A detecção e reconhecimento de objetos em ambiente não controlado tem aplicações diversas no campo da visão computacional, e juntamente com o georreferenciamento de objetos de forma automática propicia uma variedade de aplicações, como por exemplo, o mapeamento da sinalização de trânsito. Os sinais de trânsito são muito importantes por proverem regras de navegação nas ruas e estradas. Um sistema para a determinação das posições geográficas de placas de sinalização de trânsito em áreas urbanas de forma automática constitui uma ferramenta útil para a gestão municipal podendo servir para tomadas de decisão, como por exemplo, fluxo de tráfego e definição de sinalização nas vias terrestres. Do ponto de vista prático, um sistema com estas características tem uma grande complexidade na implementação o que caracteriza um grande desafio. Diante do contexto exposto, nesta tese, é tratada a computação da detecção, o reconhecimento de sinais e o georreferenciamento de placas de trânsito. A implementação deste trabalho consistiu na coleta de conjuntos de dados e a aplicação de algoritmos para a extração dos descritores de pontos chave e para realizar a correspondências dos pontos chave entre duas imagens (imagem de uma via contendo uma ou mais placas e imagem de um template de uma placa de sinalização). Uma vez obtidos apenas os pontos em comuns referentes aos seus descritores, na sequência foram aplicados algoritmos para a detecção, reconhecimento e georreferenciamento das placas de trânsito. Para a obtenção do conjunto de dados foi utilizado um sistema móvel de mapeamento terrestre, equipado com sensores de imageamento digital, que além de obter conjuntos de sequências de imagens, também capturam informações de navegação e posicionamento. Para a detecção e reconhecimento foram utilizados algoritmos já consolidados na literatura (SIFT e BBF) e também algoritmos definidos e implementados para a realização da metodologia proposta. Para a extração de pontos chave condizentes com a placa a ser detectada, foi desenvolvido um algoritmo, pelo fato dos algoritmos citados na literatura não serem adequados para imagens que apresentam poucos pontos de correspondência, como é o caso do algoritmo RANSAC. Foi também definido e implementado um algoritmo para o reconhecimento de caracteres para o caso de placas de sinalização que especificam limite de velocidade. Com o conhecimento das fotocoordenadas centrais referentes às placas detectadas e reconhecidas e os dados de navegação e posicionamento, é realizado o georreferenciamento a fim de determinar as posições das placas no terreno por meio das equações de colinearidade. Foram realizados experimentos iniciais comprovando que a metodologia proposta é adequada para os objetivos definidos. As taxas de acerto na detecção e reconhecimento das placas de sinalização atingiram valores superiores a 80%, mesmo utilizando imagens com cenas complexas. O trabalho desenvolvido contribui com a metodologia proposta destinada à determinação das posições das feições dos sinais de trânsito em áreas urbanas, e na Área de Visão Computacional, contribui com novos algoritmos para a detecção e reconhecimento de placas de sinalização, bem como um novo algoritmo para o reconhecimento de caracteres. / The detection and object recognition in uncontrolled environment has several applications in the field of computer vision, and together with automatic georeferencing of objects provides a variety of applications, for example, the mapping of traffic signs. Traffic signs are very important because they provide navigation rules in streets and roads. A system for the automatic determining of the geographic positions of traffic sign plates in urban areas constitutes a useful tool for municipal management, it can be used for decision making, such as traffic flow and sign location on roads. From a practical point of view, a system with these characteristics has a great complexity in the implementation that characterizes a great challenge. Considering the exposed context, this thesis treats the computation of detection, recognition and georeferencing of traffic signs. The implementation of this work consisted in collecting data sets and application of algorithms for extracting keypoint features and performing the keypoint matching between two images (image of a road containing one or more plates and image of a template from a traffic sign). Once only the points in common in relation to their descriptors had been obtained, in the sequence, some algorithms were applied to the detection, recognition and georeferencing of traffic signs. To obtain the data set a landbase mobile mapping system was used, equipped with digital imaging sensors, which in addition to obtaining sets of image sequences, they also capture navigation information and positioning. For detection and recognition algorithms already established in literature (SIFT and BBF) were used and algorithms defined and implemented to the realization of the proposed methodology were also used. For the extraction of keypoints suitable with the plateto be detected, an algorithm was developed, because of the algorithms mentioned in literature are not appropriate for images that have few points of matching such as the RANSAC algorithm. An algorithm for recognition of characters for the case of signs which specify the speed limit was also defined and implemented. With the knowledge of the central photo coordinates referring to plates detected and recognized and navigation and positioning data,the georeferencing is performed to determine the positions of the plates on the ground through the collinearity equations. Initial experiments were performed demonstrating that the proposed methodology is appropriate for the defined goals. The hit rates of detection and recognition of sign plates reached values above 80%, even using images with complex scenes. The developed work contributes with the proposed methodology destined to the determination of traffic signs positions in urban areas, and in the Computer Vision Area, it contributes with new algorithms for the detection and recognition of traffic signs and a new algorithm for character recognition.

Apesar da popularidade das câmeras onidirecionais aplicadas à robótica móvel e da importância do reconhecimento de objetos no universo mais amplo da robótica e da visão computacional, é difícil encontrar trabalhos que relacionem ambos na literatura especializada. Este trabalho visa avaliar um método para classificação de objetos em imagens onidirecionais, analisando sua eficácia e eficiência para ser aplicado em tarefas de auto-localização e mapeamento de ambientes feitas por robôs moveis. Tal método é construído a partir de um classificador de objetos, implementado através de máquinas de suporte vetorial, estendidas para a utilização de Aprendizagem de Múltiplos Núcleos. Também na construção deste método, uma etapa de retificação é aplicada às imagens onidirecionais, de modo a aproximá-las das imagens convencionais, às quais o classificador utilizado já demonstrou bons resultados. A abordagem de Múltiplos Núcleos se faz necessária para possibilitar a aplicação de três tipos distintos de detectores de características em imagens, ponderando, para cada classe, a importância de cada uma das características em sua descrição. Resultados experimentais atestam a viabilidade de tal proposta. / Despite the popularity of omnidirectional cameras used in mobile robotics, and the importance of object recognition in the broader universe of robotics and computer vision, it is difficult to find works that relate both in the literature. This work aims at performing the evaluation of a method for object classification in omnidirectional images, evaluating its effectiveness and efficience considering its application to tasks of self-localization and environment mapping made by mobile robots. The method is based on a multiple kernel learning extended support vector machine object classifier. Furthermore, an unwrapping step is applied to omnidirectional images, to make them similar to perspective images, to which the classifier used has already shown good results. The Multiple Kernels approach is necessary to allow the use of three distinct types of feature detectors in omnidirectional images by considering, for each class, the importance of each feature in the description. Experimental results demonstrate the feasibility of such a proposal.

A detecção e reconhecimento de objetos em ambiente não controlado tem aplicações diversas no campo da visão computacional, e juntamente com o georreferenciamento de objetos de forma automática propicia uma variedade de aplicações, como por exemplo, o mapeamento da sinalização de trânsito. Os sinais de trânsito são muito importantes por proverem regras de navegação nas ruas e estradas. Um sistema para a determinação das posições geográficas de placas de sinalização de trânsito em áreas urbanas de forma automática constitui uma ferramenta útil para a gestão municipal podendo servir para tomadas de decisão, como por exemplo, fluxo de tráfego e definição de sinalização nas vias terrestres. Do ponto de vista prático, um sistema com estas características tem uma grande complexidade na implementação o que caracteriza um grande desafio. Diante do contexto exposto, nesta tese, é tratada a computação da detecção, o reconhecimento de sinais e o georreferenciamento de placas de trânsito. A implementação deste trabalho consistiu na coleta de conjuntos de dados e a aplicação de algoritmos para a extração dos descritores de pontos chave e para realizar a correspondências dos pontos chave entre duas imagens (imagem de uma via contendo uma ou mais placas e imagem de um template de uma placa de sinalização). Uma vez obtidos apenas os pontos em comuns referentes aos seus descritores, na sequência foram aplicados algoritmos para a detecção, reconhecimento e georreferenciamento das placas de trânsito. Para a obtenção do conjunto de dados foi utilizado um sistema móvel de mapeamento terrestre, equipado com sensores de imageamento digital, que além de obter conjuntos de sequências de imagens, também capturam informações de navegação e posicionamento. Para a detecção e reconhecimento foram utilizados algoritmos já consolidados na literatura (SIFT e BBF) e também algoritmos definidos e implementados para a realização da metodologia proposta. Para a extração de pontos chave condizentes com a placa a ser detectada, foi desenvolvido um algoritmo, pelo fato dos algoritmos citados na literatura não serem adequados para imagens que apresentam poucos pontos de correspondência, como é o caso do algoritmo RANSAC. Foi também definido e implementado um algoritmo para o reconhecimento de caracteres para o caso de placas de sinalização que especificam limite de velocidade. Com o conhecimento das fotocoordenadas centrais referentes às placas detectadas e reconhecidas e os dados de navegação e posicionamento, é realizado o georreferenciamento a fim de determinar as posições das placas no terreno por meio das equações de colinearidade. Foram realizados experimentos iniciais comprovando que a metodologia proposta é adequada para os objetivos definidos. As taxas de acerto na detecção e reconhecimento das placas de sinalização atingiram valores superiores a 80%, mesmo utilizando imagens com cenas complexas. O trabalho desenvolvido contribui com a metodologia proposta destinada à determinação das posições das feições dos sinais de trânsito em áreas urbanas, e na Área de Visão Computacional, contribui com novos algoritmos para a detecção e reconhecimento de placas de sinalização, bem como um novo algoritmo para o reconhecimento de caracteres. / The detection and object recognition in uncontrolled environment has several applications in the field of computer vision, and together with automatic georeferencing of objects provides a variety of applications, for example, the mapping of traffic signs. Traffic signs are very important because they provide navigation rules in streets and roads. A system for the automatic determining of the geographic positions of traffic sign plates in urban areas constitutes a useful tool for municipal management, it can be used for decision making, such as traffic flow and sign location on roads. From a practical point of view, a system with these characteristics has a great complexity in the implementation that characterizes a great challenge. Considering the exposed context, this thesis treats the computation of detection, recognition and georeferencing of traffic signs. The implementation of this work consisted in collecting data sets and application of algorithms for extracting keypoint features and performing the keypoint matching between two images (image of a road containing one or more plates and image of a template from a traffic sign). Once only the points in common in relation to their descriptors had been obtained, in the sequence, some algorithms were applied to the detection, recognition and georeferencing of traffic signs. To obtain the data set a landbase mobile mapping system was used, equipped with digital imaging sensors, which in addition to obtaining sets of image sequences, they also capture navigation information and positioning. For detection and recognition algorithms already established in literature (SIFT and BBF) were used and algorithms defined and implemented to the realization of the proposed methodology were also used. For the extraction of keypoints suitable with the plateto be detected, an algorithm was developed, because of the algorithms mentioned in literature are not appropriate for images that have few points of matching such as the RANSAC algorithm. An algorithm for recognition of characters for the case of signs which specify the speed limit was also defined and implemented. With the knowledge of the central photo coordinates referring to plates detected and recognized and navigation and positioning data,the georeferencing is performed to determine the positions of the plates on the ground through the collinearity equations. Initial experiments were performed demonstrating that the proposed methodology is appropriate for the defined goals. The hit rates of detection and recognition of sign plates reached values above 80%, even using images with complex scenes. The developed work contributes with the proposed methodology destined to the determination of traffic signs positions in urban areas, and in the Computer Vision Area, it contributes with new algorithms for the detection and recognition of traffic signs and a new algorithm for character recognition.

This dissertation develops a novel system for object recognition in videos. The input of the system is a set of unconstrained videos containing a known set of objects. The output is the locations and categories for each object in each frame across all videos. Initially, a shot boundary detection algorithm is applied to the videos to divide them into multiple sequences separated by the identified shot boundaries. Since each of these sequences still contains moderate content variations, we further use a cost optimization-based key frame extraction method to select key frames in each sequence and use these key frames to divide the videos into shorter sub-sequences with little content variations. Next, we learn object proposals on the first frame of each sub-sequence. Building upon the state-of-the-art object detection algorithms, we develop a tree-based hierarchical model to improve the object detection. Using the learned object proposals as the initial object positions in the first frame of each sub-sequence, we apply the SPOT tracker to track the object proposals and re-rank them using the proposed temporal objectness to obtain object proposals tubes by removing unlikely objects. Finally, we employ the deep Convolution Neural Network (CNN) to perform classification on these tubes. Experiments show that the proposed system significantly improves the object detection rate of the learned proposals when comparing with some state-of-the-art object detectors. Due to the improvement in object detection, the proposed system also achieves higher mean average precision at the stage of proposal classification than the state-of-the-art methods.

<p>Shape-based matching (SBM) is a method for matching objects in greyscale images. It extracts edges from search images and matches them to a model using a similarity measure. In this thesis we extend SBM to find the tilt and height position of the object in addition to the z-plane rotation and x-y-position. The search is conducted using a scale pyramid to improve the search speed. A 3D matching can be done for small tilt angles by using SBM on height data and extending it with additional steps to calculate the tilt of the object. The full pose is useful for picking objects with an industrial robot.</p><p>The tilt of the object is calculated using a RANSAC plane estimator. After the 2D search the differences in height between all corresponding points of the model and the live image are calculated. By estimating a plane to this difference the tilt of the object can be calculated. Using the tilt the model edges are tilted in order to improve the matching at the next scale level.</p><p>The problems that arise with occlusion and missing data have been studied. Missing data and erroneous data have been thresholded manually after conducting tests where automatic filling of missing data did not noticeably improve the matching. The automatic filling could introduce new false edges and remove true ones, thus lowering the score.</p><p>Experiments have been conducted where objects have been placed at increasing tilt angles. The results show that the matching algorithm is object dependent and correct matches are almost always found for tilt angles less than 10 degrees. This is very similar to the original 2D SBM because the model edges does not change much for such small angels. For tilt angles up to about 25 degrees most objects can be matched and for nice objects correct matches can be done at large tilt angles of up to 40 degrees.</p>

This dissertation addresses a number of inter-related and fundamental problems in computer vision. Specifically, we address object discovery, recognition, segmentation, and 3D pose estimation in images, as well as 3D scene reconstruction and scene interpretation. The key ideas behind our approaches include using shape as a basic object feature, and using structured prediction modeling paradigms for representing objects and scenes. In this work, we make a number of new contributions both in computer vision and machine learning. We address the vision problems of shape matching, shape-based mining of objects in arbitrary image collections, context-aware object recognition, monocular estimation of 3D object poses, and monocular 3D scene reconstruction using shape from texture. Our work on shape-based object discovery is the first to show that meaningful objects can be extracted from a collection of arbitrary images, without any human supervision, by shape matching. We also show that a spatial repetition of objects in images (e.g., windows on a building facade, or cars lined up along a street) can be used for 3D scene reconstruction from a single image. The aforementioned topics have never been addressed in the literature. The dissertation also presents new algorithms and object representations for the aforementioned vision problems. We fuse two traditionally different modeling paradigms Conditional Random Fields (CRF) and Random Forests (RF) into a unified framework, referred to as (RF)^2. We also derive theoretical error bounds of estimating distribution ratios by a two-class RF, which is then used to derive the theoretical performance bounds of a two-class (RF)^2. Thorough experimental evaluation of individual aspects of all our approaches is presented. In general, the experiments demonstrate that we outperform the state of the art on the benchmark datasets, without increasing complexity and supervision in training. / Graduation date: 2011 / Access restricted to the OSU Community at author's request from May 12, 2011 - May 12, 2012

This paper continues our work on visual representation s of three-dimensional surfaces [Brady and Yuille 1984b]. The theoretical component of our work is a study of classes of surface curves as a source of constraint n the surface on which they lie, and as a basis for describing it. We analyze bounding contours, surface intersections, lines of curvature, and asymptotes. Our experimental work investigates whether the information suggested by our theoretical study can be computed reliably and efficiently. We demonstrate algorithms that compute lines of curvature of a (Gaussian smoothed) surface; determine planar patches and umbilic regions; extract axes of surfaces of revolution and tube surfaces. We report preliminary results on adapting the curvature primal sketch algorithms of Asada and Brady [1984] to detect and describe surface intersections.

Alignment is a prevalent approach for recognizing 3D objects in 2D images. A major problem with current implementations is how to robustly handle errors that propagate from uncertainties in the locations of image features. This thesis gives a technique for bounding these errors. The technique makes use of a new solution to the problem of recovering 3D pose from three matching point pairs under weak-perspective projection. Furthermore, the error bounds are used to demonstrate that using line segments for features instead of points significantly reduces the false positive rate, to the extent that alignment can remain reliable even in cluttered scenes.

We describe a method for modeling object classes (such as faces) using 2D example images and an algorithm for matching a model to a novel image. The object class models are "learned'' from example images that we call prototypes. In addition to the images, the pixelwise correspondences between a reference prototype and each of the other prototypes must also be provided. Thus a model consists of a linear combination of prototypical shapes and textures. A stochastic gradient descent algorithm is used to match a model to a novel image by minimizing the error between the model and the novel image. Example models are shown as well as example matches to novel images. The robustness of the matching algorithm is also evaluated. The technique can be used for a number of applications including the computation of correspondence between novel images of a certain known class, object recognition, image synthesis and image compression.

We present an example-based learning approach for locating vertical frontal views of human faces in complex scenes. The technique models the distribution of human face patterns by means of a few view-based "face'' and "non-face'' prototype clusters. At each image location, the local pattern is matched against the distribution-based model, and a trained classifier determines, based on the local difference measurements, whether or not a human face exists at the current image location. We provide an analysis that helps identify the critical components of our system.