A detecção e reconhecimento de objetos em ambiente não controlado tem aplicações diversas no campo da visão computacional, e juntamente com o georreferenciamento de objetos de forma automática propicia uma variedade de aplicações, como por exemplo, o mapeamento da sinalização de trânsito. Os sinais de trânsito são muito importantes por proverem regras de navegação nas ruas e estradas. Um sistema para a determinação das posições geográficas de placas de sinalização de trânsito em áreas urbanas de forma automática constitui uma ferramenta útil para a gestão municipal podendo servir para tomadas de decisão, como por exemplo, fluxo de tráfego e definição de sinalização nas vias terrestres. Do ponto de vista prático, um sistema com estas características tem uma grande complexidade na implementação o que caracteriza um grande desafio. Diante do contexto exposto, nesta tese, é tratada a computação da detecção, o reconhecimento de sinais e o georreferenciamento de placas de trânsito. A implementação deste trabalho consistiu na coleta de conjuntos de dados e a aplicação de algoritmos para a extração dos descritores de pontos chave e para realizar a correspondências dos pontos chave entre duas imagens (imagem de uma via contendo uma ou mais placas e imagem de um template de uma placa de sinalização). Uma vez obtidos apenas os pontos em comuns referentes aos seus descritores, na sequência foram aplicados algoritmos para a detecção, reconhecimento e georreferenciamento das placas de trânsito. Para a obtenção do conjunto de dados foi utilizado um sistema móvel de mapeamento terrestre, equipado com sensores de imageamento digital, que além de obter conjuntos de sequências de imagens, também capturam informações de navegação e posicionamento. Para a detecção e reconhecimento foram utilizados algoritmos já consolidados na literatura (SIFT e BBF) e também algoritmos definidos e implementados para a realização da metodologia proposta. Para a extração de pontos chave condizentes com a placa a ser detectada, foi desenvolvido um algoritmo, pelo fato dos algoritmos citados na literatura não serem adequados para imagens que apresentam poucos pontos de correspondência, como é o caso do algoritmo RANSAC. Foi também definido e implementado um algoritmo para o reconhecimento de caracteres para o caso de placas de sinalização que especificam limite de velocidade. Com o conhecimento das fotocoordenadas centrais referentes às placas detectadas e reconhecidas e os dados de navegação e posicionamento, é realizado o georreferenciamento a fim de determinar as posições das placas no terreno por meio das equações de colinearidade. Foram realizados experimentos iniciais comprovando que a metodologia proposta é adequada para os objetivos definidos. As taxas de acerto na detecção e reconhecimento das placas de sinalização atingiram valores superiores a 80%, mesmo utilizando imagens com cenas complexas. O trabalho desenvolvido contribui com a metodologia proposta destinada à determinação das posições das feições dos sinais de trânsito em áreas urbanas, e na Área de Visão Computacional, contribui com novos algoritmos para a detecção e reconhecimento de placas de sinalização, bem como um novo algoritmo para o reconhecimento de caracteres. / The detection and object recognition in uncontrolled environment has several applications in the field of computer vision, and together with automatic georeferencing of objects provides a variety of applications, for example, the mapping of traffic signs. Traffic signs are very important because they provide navigation rules in streets and roads. A system for the automatic determining of the geographic positions of traffic sign plates in urban areas constitutes a useful tool for municipal management, it can be used for decision making, such as traffic flow and sign location on roads. From a practical point of view, a system with these characteristics has a great complexity in the implementation that characterizes a great challenge. Considering the exposed context, this thesis treats the computation of detection, recognition and georeferencing of traffic signs. The implementation of this work consisted in collecting data sets and application of algorithms for extracting keypoint features and performing the keypoint matching between two images (image of a road containing one or more plates and image of a template from a traffic sign). Once only the points in common in relation to their descriptors had been obtained, in the sequence, some algorithms were applied to the detection, recognition and georeferencing of traffic signs. To obtain the data set a landbase mobile mapping system was used, equipped with digital imaging sensors, which in addition to obtaining sets of image sequences, they also capture navigation information and positioning. For detection and recognition algorithms already established in literature (SIFT and BBF) were used and algorithms defined and implemented to the realization of the proposed methodology were also used. For the extraction of keypoints suitable with the plateto be detected, an algorithm was developed, because of the algorithms mentioned in literature are not appropriate for images that have few points of matching such as the RANSAC algorithm. An algorithm for recognition of characters for the case of signs which specify the speed limit was also defined and implemented. With the knowledge of the central photo coordinates referring to plates detected and recognized and navigation and positioning data,the georeferencing is performed to determine the positions of the plates on the ground through the collinearity equations. Initial experiments were performed demonstrating that the proposed methodology is appropriate for the defined goals. The hit rates of detection and recognition of sign plates reached values above 80%, even using images with complex scenes. The developed work contributes with the proposed methodology destined to the determination of traffic signs positions in urban areas, and in the Computer Vision Area, it contributes with new algorithms for the detection and recognition of traffic signs and a new algorithm for character recognition.

A detecção e reconhecimento de objetos em ambiente não controlado tem aplicações diversas no campo da visão computacional, e juntamente com o georreferenciamento de objetos de forma automática propicia uma variedade de aplicações, como por exemplo, o mapeamento da sinalização de trânsito. Os sinais de trânsito são muito importantes por proverem regras de navegação nas ruas e estradas. Um sistema para a determinação das posições geográficas de placas de sinalização de trânsito em áreas urbanas de forma automática constitui uma ferramenta útil para a gestão municipal podendo servir para tomadas de decisão, como por exemplo, fluxo de tráfego e definição de sinalização nas vias terrestres. Do ponto de vista prático, um sistema com estas características tem uma grande complexidade na implementação o que caracteriza um grande desafio. Diante do contexto exposto, nesta tese, é tratada a computação da detecção, o reconhecimento de sinais e o georreferenciamento de placas de trânsito. A implementação deste trabalho consistiu na coleta de conjuntos de dados e a aplicação de algoritmos para a extração dos descritores de pontos chave e para realizar a correspondências dos pontos chave entre duas imagens (imagem de uma via contendo uma ou mais placas e imagem de um template de uma placa de sinalização). Uma vez obtidos apenas os pontos em comuns referentes aos seus descritores, na sequência foram aplicados algoritmos para a detecção, reconhecimento e georreferenciamento das placas de trânsito. Para a obtenção do conjunto de dados foi utilizado um sistema móvel de mapeamento terrestre, equipado com sensores de imageamento digital, que além de obter conjuntos de sequências de imagens, também capturam informações de navegação e posicionamento. Para a detecção e reconhecimento foram utilizados algoritmos já consolidados na literatura (SIFT e BBF) e também algoritmos definidos e implementados para a realização da metodologia proposta. Para a extração de pontos chave condizentes com a placa a ser detectada, foi desenvolvido um algoritmo, pelo fato dos algoritmos citados na literatura não serem adequados para imagens que apresentam poucos pontos de correspondência, como é o caso do algoritmo RANSAC. Foi também definido e implementado um algoritmo para o reconhecimento de caracteres para o caso de placas de sinalização que especificam limite de velocidade. Com o conhecimento das fotocoordenadas centrais referentes às placas detectadas e reconhecidas e os dados de navegação e posicionamento, é realizado o georreferenciamento a fim de determinar as posições das placas no terreno por meio das equações de colinearidade. Foram realizados experimentos iniciais comprovando que a metodologia proposta é adequada para os objetivos definidos. As taxas de acerto na detecção e reconhecimento das placas de sinalização atingiram valores superiores a 80%, mesmo utilizando imagens com cenas complexas. O trabalho desenvolvido contribui com a metodologia proposta destinada à determinação das posições das feições dos sinais de trânsito em áreas urbanas, e na Área de Visão Computacional, contribui com novos algoritmos para a detecção e reconhecimento de placas de sinalização, bem como um novo algoritmo para o reconhecimento de caracteres. / The detection and object recognition in uncontrolled environment has several applications in the field of computer vision, and together with automatic georeferencing of objects provides a variety of applications, for example, the mapping of traffic signs. Traffic signs are very important because they provide navigation rules in streets and roads. A system for the automatic determining of the geographic positions of traffic sign plates in urban areas constitutes a useful tool for municipal management, it can be used for decision making, such as traffic flow and sign location on roads. From a practical point of view, a system with these characteristics has a great complexity in the implementation that characterizes a great challenge. Considering the exposed context, this thesis treats the computation of detection, recognition and georeferencing of traffic signs. The implementation of this work consisted in collecting data sets and application of algorithms for extracting keypoint features and performing the keypoint matching between two images (image of a road containing one or more plates and image of a template from a traffic sign). Once only the points in common in relation to their descriptors had been obtained, in the sequence, some algorithms were applied to the detection, recognition and georeferencing of traffic signs. To obtain the data set a landbase mobile mapping system was used, equipped with digital imaging sensors, which in addition to obtaining sets of image sequences, they also capture navigation information and positioning. For detection and recognition algorithms already established in literature (SIFT and BBF) were used and algorithms defined and implemented to the realization of the proposed methodology were also used. For the extraction of keypoints suitable with the plateto be detected, an algorithm was developed, because of the algorithms mentioned in literature are not appropriate for images that have few points of matching such as the RANSAC algorithm. An algorithm for recognition of characters for the case of signs which specify the speed limit was also defined and implemented. With the knowledge of the central photo coordinates referring to plates detected and recognized and navigation and positioning data,the georeferencing is performed to determine the positions of the plates on the ground through the collinearity equations. Initial experiments were performed demonstrating that the proposed methodology is appropriate for the defined goals. The hit rates of detection and recognition of sign plates reached values above 80%, even using images with complex scenes. The developed work contributes with the proposed methodology destined to the determination of traffic signs positions in urban areas, and in the Computer Vision Area, it contributes with new algorithms for the detection and recognition of traffic signs and a new algorithm for character recognition.

<p>The field of autonomous vehicle navigation and localization is a highly active research</p><p>topic. The aim of this thesis is to evaluate the feasibility to use outdoor visual navigation in a semi-structured environment. The goal is to develop a visual navigation system for an autonomous golf ball collection vehicle operating on driving ranges.</p><p>The image feature extractors SIFT and PCA-SIFT was evaluated on an image database</p><p>consisting of images acquired from 19 outdoor locations over a period of several weeks to</p><p>allow different environmental conditions. The results from these tests show that SIFT-type</p><p>feature extractors are able to find and match image features with high accuracy. The results also show that this can be improved further by a combination of a lower nearest neighbour threshold and an outlier rejection method to allow more matches and a higher ratio of correct matches. Outliers were found and rejected by fitting the data to a homography model with the RANSAC robust estimator algorithm. </p><p>A simulator was developed to evaluate the suggested system with respect to pixel noise from illumination changes, weather and feature position accuracy as well as the distance to features, path shapes and the visual servoing target image (milestone) interval. The system was evaluated on a total of 3 paths, 40 test combinations and 137km driven. The results show that with the relatively simple visual servoing navigation system it is possible to use mono-vision as a sole sensor and navigate semi-structured outdoor environments such as driving ranges.</p>

The field of autonomous vehicle navigation and localization is a highly active research topic. The aim of this thesis is to evaluate the feasibility to use outdoor visual navigation in a semi-structured environment. The goal is to develop a visual navigation system for an autonomous golf ball collection vehicle operating on driving ranges. The image feature extractors SIFT and PCA-SIFT was evaluated on an image database consisting of images acquired from 19 outdoor locations over a period of several weeks to allow different environmental conditions. The results from these tests show that SIFT-type feature extractors are able to find and match image features with high accuracy. The results also show that this can be improved further by a combination of a lower nearest neighbour threshold and an outlier rejection method to allow more matches and a higher ratio of correct matches. Outliers were found and rejected by fitting the data to a homography model with the RANSAC robust estimator algorithm. A simulator was developed to evaluate the suggested system with respect to pixel noise from illumination changes, weather and feature position accuracy as well as the distance to features, path shapes and the visual servoing target image (milestone) interval. The system was evaluated on a total of 3 paths, 40 test combinations and 137km driven. The results show that with the relatively simple visual servoing navigation system it is possible to use mono-vision as a sole sensor and navigate semi-structured outdoor environments such as driving ranges.

Conselho Nacional de Desenvolvimento Científico e Tecnológico / Neste trabalho é estudada a viabilidade de uma implementação em paralelo do algoritmo scale invariant feature transform (SIFT) para identificação de íris. Para a implementação do código foi utilizada a arquitetura para computação paralela compute unified device architecture (CUDA) e a linguagem OpenGL shading language (GLSL). O algoritmo foi testado utilizando três bases de dados de olhos e íris, o noisy visible wavelength iris image Database (UBIRIS), Michal-Libor e CASIA. Testes foram feitos para determinar o tempo de processamento para verificação da presença ou não de um indivíduo em um banco de dados, determinar a eficiência dos algoritmos de busca implementados em GLSL e CUDA e buscar valores de calibração que melhoram o posicionamento e a distribuição dos pontos-chave na região de interesse (íris) e a robustez do programa final. / Present work studies the feasibility of a parallel implementation of the scene recognition algorithm SIFT for iris recognition. The code was built using the Compute Unified Device Architecture (CUDA) and the shading language GLSL. The algorithm was tested using three databases containing eyes and iris, the UBIRIS, Michal- Libor and CASIA. Tests were made for: analyzing the requested time for checking if an subject is or is not present on current database, the efficiency of the search algorithms written in CUDA and GLSL, the search for calibration values that improve keypoints position and distribution through the region of interest (iris), analyzing the reliability of the final code.

[pt] A partir da disponibilização comercial de imagens de alta resolução, modelos 3D de superfícies geradas a partir de imagens aéreas e de satélite tornaram-se uma alternativa mais atraente para aplicações como planejamento de telecomunicações, monitoramento de desastres e planejamento urbano. A exatidão dos modelos 3D da superfície terrestre baseados em pares de imagens estereoscópicas depende da exatidão com que pontos homólogos são localizados em ambas as imagens. Os métodos automáticos de localização de pontos homólogos em imagens digitais baseados em área, combinados com técnicas de crescimento de região, são capazes de produzir uma nuvem densa e exata de pontos homólogos. Entretanto, o processo de crescimento de região pode ser interrompido em regiões da imagem cujo efeito de uma variação abrupta da paralaxe no eixo x aparece de maneira importante. Neste caso, novas sementes devem ser introduzidas, normalmente por um operador humano. A partir dessas novas sementes, o processo será reiniciado. Dependendo do tipo da imagem utilizada e da estrutura 3D da região mapeada, a intervenção humana pode ser considerável. Propõe-se então uma alternativa completamente automatizada no qual se combinam as técnicas do SIFT (Scale Invariant Feature Transform), mínimos quadrados e crescimento de região. Experimentos realizados em pares de imagens aéreas e de satélite cobrindo diversos tipos de terrenos mostraram a eficácia do método proposto, especialmente em regiões com mudanças abruptas de elevação, como fachadas de prédios altos. / [en] After the high resolution images became commercially available, 3D surface models generated from space-born stereo images turned into an attractive alternative for applications such as telecommunication planning, disaster monitoring and urban planning. The accuracy of the 3D models of the earth surface depends on the accuracy of corresponding points located in both images. Area-based automatic image matching combined with a region-growing technique are able to provide a dense and accurate grid of corresponding points. However the region-growing process may stop at image patches where the effect of a sudden change in the x-parallax is important. In such a case new seed points must be provided, usually by human operator. From the additional seed points the region-growing procedure may continue. Depending upon the type of image and the 3D-structure of the mapped area, the human intervention may be considerable. A fully automatic alternative that combines the SIFT (Scale Invariant Feature Transform), least square matching and region-growing technique is proposed in this work. Experiments conducted on stereo pairs of Ikonos and aerial images covering different terrain types have shown the effectiveness of the proposed method especially in locations with abrupt height changes, such as façades of high buildings.

Conselho Nacional de Desenvolvimento Científico e Tecnológico / Neste trabalho é estudada a viabilidade de uma implementação em paralelo do algoritmo scale invariant feature transform (SIFT) para identificação de íris. Para a implementação do código foi utilizada a arquitetura para computação paralela compute unified device architecture (CUDA) e a linguagem OpenGL shading language (GLSL). O algoritmo foi testado utilizando três bases de dados de olhos e íris, o noisy visible wavelength iris image Database (UBIRIS), Michal-Libor e CASIA. Testes foram feitos para determinar o tempo de processamento para verificação da presença ou não de um indivíduo em um banco de dados, determinar a eficiência dos algoritmos de busca implementados em GLSL e CUDA e buscar valores de calibração que melhoram o posicionamento e a distribuição dos pontos-chave na região de interesse (íris) e a robustez do programa final. / Present work studies the feasibility of a parallel implementation of the scene recognition algorithm SIFT for iris recognition. The code was built using the Compute Unified Device Architecture (CUDA) and the shading language GLSL. The algorithm was tested using three databases containing eyes and iris, the UBIRIS, Michal- Libor and CASIA. Tests were made for: analyzing the requested time for checking if an subject is or is not present on current database, the efficiency of the search algorithms written in CUDA and GLSL, the search for calibration values that improve keypoints position and distribution through the region of interest (iris), analyzing the reliability of the final code.

Master of Science / Department of Computing and Information Sciences / David A. Gustafson William Hsu / In this study, we identify a new method of tracking motion over a sequence of images using feature point clusters. We identify and implement a system that takes as input a sequence of images and generates clusters of SIFT features using the K-Means clustering algorithm. Every time the system processes an image it compares each new cluster to the clusters of previous images, which it stores in a local cache. When at least 25% of the SIFT features that compose a cluster match a cluster in the local cache, the system uses the centroid of both clusters in order to determine the direction of travel. To establish a direction of travel, we calculate the slope of the line connecting the centroid of two clusters relative to their Cartesian coordinates in the secondary image. In an experiment using a P3-AT mobile robotic agent equipped with a digital camera, the system receives and processes a sequence of eight images. Experimental results show that the system is able to identify and track the motion of objects using SIFT feature clusters more efficiently when applying spatial outlier detection prior to generating clusters.

The rapid growth of digital video content in recent years has imposed the need for the development of technologies with the capability to produce condensed but semantically rich versions of the input video stream in an effective manner. Consequently, the topic of Video Summarisation is becoming increasingly popular in multimedia community and numerous video abstraction approaches have been proposed accordingly. These recommended techniques can be divided into two major categories of automatic and semi-automatic in accordance with the required level of human intervention in summarisation process. The fully-automated methods mainly adopt the low-level visual, aural and textual features alongside the mathematical and statistical algorithms in furtherance to extract the most significant segments of original video. However, the effectiveness of this type of techniques is restricted by a number of factors such as domain-dependency, computational expenses and the inability to understand the semantics of videos from low-level features. The second category of techniques however, attempts to alleviate the quality of summaries by involving humans in the abstraction process to bridge the semantic gap. Nonetheless, a single user’s subjectivity and other external contributing factors such as distraction will potentially deteriorate the performance of this group of approaches. Accordingly, in this thesis we have focused on the development of three user-centred effective video summarisation techniques that could be applied to different video categories and generate satisfactory results. According to our first proposed approach, a novel mechanism for a user-centred video summarisation has been presented for the scenarios in which multiple actors are employed in the video summarisation process in order to minimise the negative effects of sole user adoption. Based on our recommended algorithm, the video frames were initially scored by a group of video annotators ‘on the fly’. This was followed by averaging these assigned scores in order to generate a singular saliency score for each video frame and, finally, the highest scored video frames alongside the corresponding audio and textual contents were extracted to be included into the final summary. The effectiveness of our approach has been assessed by comparing the video summaries generated based on our approach against the results obtained from three existing automatic summarisation tools that adopt different modalities for abstraction purposes. The experimental results indicated that our proposed method is capable of delivering remarkable outcomes in terms of Overall Satisfaction and Precision with an acceptable Recall rate, indicating the usefulness of involving user input in the video summarisation process. In an attempt to provide a better user experience, we have proposed our personalised video summarisation method with an ability to customise the generated summaries in accordance with the viewers’ preferences. Accordingly, the end-user’s priority levels towards different video scenes were captured and utilised for updating the average scores previously assigned by the video annotators. Finally, our earlier proposed summarisation method was adopted to extract the most significant audio-visual content of the video. Experimental results indicated the capability of this approach to deliver superior outcomes compared with our previously proposed method and the three other automatic summarisation tools. Finally, we have attempted to reduce the required level of audience involvement for personalisation purposes by proposing a new method for producing personalised video summaries. Accordingly, SIFT visual features were adopted to identify the video scenes’ semantic categories. Fusing this retrieved data with pre-built users’ profiles, personalised video abstracts can be created. Experimental results showed the effectiveness of this method in delivering superior outcomes comparing to our previously recommended algorithm and the three other automatic summarisation techniques.

Given a set of potential explanatory variables, one model selection approach is to select the best model, according to some criterion, from among the collection of models defined by all possible subsets of the explanatory variables. A popular procedure that has been used in this setting is to select the model that results in the smallest value of the Akaike information criterion (AIC). One drawback in using the AIC is that it can lead to the frequent selection of overspecified models. This can be problematic if the researcher wishes to assert, with some level of certainty, the necessity of any given variable that has been selected. This thesis develops a model selection procedure that allows the researcher to nominate, a priori, the probability at which overspecified models will be selected from among all possible subsets. The procedure seeks to determine if the inclusion of each candidate variable results in a sufficiently improved fitting term, and hence is referred to as the SIFT procedure. In order to determine whether there is sufficient evidence to retain a candidate variable or not, a set of threshold values are computed. Two procedures are proposed: a naive method based on a set of restrictive assumptions; and an empirical permutation-based method. Graphical tools have also been developed to be used in conjunction with the SIFT procedure. The graphical representation of the SIFT procedure clarifies the process being undertaken. Using these tools can also assist researchers in developing a deeper understanding of the data they are analyzing. The naive and empirical SIFT methods are investigated by way of simulation under a range of conditions within the standard linear model framework. The performance of the SIFT methodology is compared with model selection by minimum AIC; minimum Bayesian Information Criterion (BIC); and backward elimination based on p-values. The SIFT procedure is found to behave as designed—asymptotically selecting those variables that characterize the underlying data generating mechanism, while limiting the selection of false or spurious variables to the desired level. The SIFT methodology offers researchers a promising new approach to model selection, whereby they are now able to control the probability of selecting an overspecified model to a level that best suits their needs.