Feature matching in images plays an important role in computer vision such as for 3D reconstruction, motion analysis, object recognition, target tracking and dynamic scene analysis. In this paper, we present a robust cooperative strategy to establish the correspondence of the contours between two uncalibrated images based on the recovered epipolar geometry. We take into account two representations of contours in image as contour points and contour chains. The method proposed in the paper is composed of the following two consecutive steps: (1) The first step uses the LMedS method to estimate the fundamental matrix based on Hartley’s 8-point algorithm, (2) The second step uses a new robust cooperative strategy to match contours. The presented approach has been tested with various real images and experimental results show that our method can produce more accurate contour correspondences. / Singapore-MIT Alliance (SMA)

This master-thesis presents an approach to track and count the number of fruit incommercial mango orchards. The algorithm is intended to enable precision agri-culture and to facilitate labour and post-harvest storage planning. The primary objective is to develop an multi-view algorithm and investigate how it can beused to mitigate the effects of visual occlusion, to improve upon estimates frommethods that use a single central or two opposite viewpoints. Fruit are detectedin images by using two classification methods: dense pixel-wise cnn and regionbased r-cnn detection. Pair-wise fruit correspondences are established between images by using geometry provided by navigation data, and lidar data is used to generate image masks for each separate tree, to isolate fruit counts to individual trees. The tracked fruit are triangulated to locate them in 3D space, and spatial statistics are calculated over whole orchard blocks. The estimated tree counts are compared to single view estimates and validated against ground truth data of 16 mango trees from a Bundaberg mango orchard in Queensland, Australia. The results show a high R2-value of 0.99335 for four hand labelled trees and a highest R2-value of 0.9165 for the machine labelled images using the r-cnn classifier forthe 16 target trees.

A estimação de posição é o resultado direto da reconstrução de cenas, um dos ramos da visão computacional. É também uma informação importante para o controle de sistemas mecatrônicos, e em especial para os sistemas robóticos autônomos. Como uma aplicação de engenharia, o desempenho de tal sistema deve ser avaliado em termos de eficiência e eficácia, medidas traduzidas respectivamente pelo custo de processamento e pela quantificação do erro. A geometria epipolar é um campo da visão computacional que fornece formalismo matemático e técnicas de reconstrução de cenas a partir de uma par de imagens, através de pontos correspondentes entre elas. Através deste formalismo é possível determinar a incerteza dos métodos de estimação de posição, que são relativamente simples e podem atingir boa precisão. Dentre os sistemas robóticos autônomos destacam-se os ROVs - do inglês \"Remotely Operated Vehicles\" - ou veículos operados remotamente, muito utilizados em tarefas submarinas, e cuja necessidade crescente de autonomia motiva o desenvolvimento de um sensor de visão com características de baixo consumo de energia, flexibilidade e inteligência. Este sensor pode consistir de uma câmera CCD e algoritmos de reconstrução de cena baseados em geometria epipolar entre imagens. Este estudo visa fornecer um comparativo de resultados práticos da estimação de posição através da geometria epipolar entre imagens, como parte da implementação de um sensor de visão para robôs autônomos. Os conceitos teóricos abordados são: geometria projetiva, modelo de câmera, geometria epipolar, matriz fundamental, reconstrução projetiva, re-construção métrica, algoritmos de determinação da matriz fundamental, algoritmos de reconstrução métrica, incerteza da matriz fundamental e complexidade computacional. Os resultados práticos baseiam-se em simulações através de imagens geradas por computador e em montagens experimentais feitas em laboratório que simulam situações práticas. O processo de estimação de posição foi realizado através da implementação em MATLAB® 6.5 dos algoritmos apresentados na parte teórica, e os resultados comparados e analisados quanto ao erro e complexidade de execução. Dentre as principais conclusões é apresentado a melhor escolha para a implementação de sensor de visão de propósito geral - o Algoritmo de 8 Pontos Correspondentes Normalizado. São apresentadas também as condições de utilização de cada método e os cuidados necessários na interpretação dos resultados. / Position estimation is the direct result of scene reconstruction, one of computer vision\'s fields. It is also an important information for the control of mechanical systems - specially the autonomous robotic systems. As an engineering application, those systems\' performance must be evaluated in terms of efficiency and effectiveness, measured by processing costs and error quantification. The epipolar geometry is a field of computer vision that supply mathematical formalism and scene reconstruction techniques that are based on the correspondences between two images. Through this formalism it is possible to stipulate the uncertainty of the position estimation methods that are relatively simple and can give good accuracy. Among the autonomous robotic systems, the ROVs - Remotely Operated Vehicles - are of special interest, mostly employed in submarine activities, and whose crescent autonomy demand motivates the development of a vision sensor of low power consumption, flexibility and intelligence. This sensor may be constructed with a CCD camera and the scene reconstruction algorithms based on epipolar geometry. This work aims to build a comparison of practical results of position estimation through epipolar geometry, as part of a vision sensor implementation for autonomous robots. The theory presented in this work comprises of: projective geometry, camera model, epipolar geometry, fundamental matrix, projective reconstruction, metric reconstruction, fundamental matrix algorithms, metric reconstruction algorithms, fundamental matrix uncertainty, and computational complexity. The practical results are based on computer generated simulations and experimental assemblies that emulate practical issues. The position estimation was carried out by MATLAB® 6.5 implementations of the algorithms analyzed in the theoretical part, and the results are compared and analyzed in respect of the error and the execution complexity. The main conclusions are that the best algorithm choice for the implementation of a general purpose vision sensor is the Normalized 8 Point Algorithm, and the usage conditions of each method, besides the special considerations that must be observed at the interpretation of the results.

Sistemas de teleconferência e telepresença são ferramentas de comunicação cada vez mais comuns. Partindo da existência de um canal de comunicação de alta capacidade, busca-se permitir visualização tridimensional realista, sensível ao ponto de vista do usuário e que mantenha a estrutura física da cena sem conhecimento prévio de sua estrutura, por meio de câmeras estéreo. A partir de pares de imagens temporalmente coerentes são sintetizadas visões intermediárias da cena alvo, de modo que um usuário rastreado tenha a ilusão de estar vendo a cena real através de uma janela. O sistema implementado baseia-se em dispositivos que provejam melhor qualidade visual no componente de geração de imagens intermediárias e procura maximizar a facilidade de uso na parte de visualização, empregando dispositivos domésticos e dispensando procedimentos complexos de calibração. / Teleconference and telepresence systems are increasingly common. Based on the existence of a high capacity communication channel, it is aimed to provide user\'s point of view sensible realistic 3D visualization without physical distortion or any prior knowledge about the structure of the scene, using stereo cameras. Pairs of temporally coherent images are used to generate intermediary view of the target scene so that a tracked user will have the illusion of posing in front so the real scene. The implemented system is based on high visual quality components on the side of views generation and tries to maximize ease of use in the visualization part by using commodity components and being free so complex calibration procedures.

This dissertation describes the design and implementation of a system that has been designed to extract 3D information from pairs of 2D images. System input consists of two images taken by an ordinary digital camera. System output is a full 3D model extracted from 2D images. There are no assumptions about the positions of the cameras during the time when the images are being taken, but the scene must not undergo any modifications. The process of extracting 3D information from 2D images consists of three basic steps. First, point matching is performed. The main contribution of this step is the introduction of an approach to matching image segments in the context of an approximation space. The second step copes with the problem of estimating external camera parameters. The proposed solution to this problem uses 3D geometry rather than the fundamental matrix widely used in 2D to 3D conversion. In the proposed approach (DirectGS), the distances between reprojected rays for all image points are minimised. The contribution of the approach considered in this step is a definition of an optimal search space for solving the 2D to 3D conversion problem and introduction of an efficient algorithm that minimises reprojection error. In the third step, the problem of dense matching is considered. The contribution of this step is the introduction of a proposed approach to dense matching of 3D object structures that utilises the presence of points on lines in 3D space. The theory and experiments developed for this dissertation demonstrate the usefulness of the proposed system in the process of digitizing 3D information. The main advantage of the proposed approach is its low cost, simplicity in use for an untrained user and the high precision of reconstructed objects. / October 2007

This dissertation describes the design and implementation of a system that has been designed to extract 3D information from pairs of 2D images. System input consists of two images taken by an ordinary digital camera. System output is a full 3D model extracted from 2D images. There are no assumptions about the positions of the cameras during the time when the images are being taken, but the scene must not undergo any modifications. The process of extracting 3D information from 2D images consists of three basic steps. First, point matching is performed. The main contribution of this step is the introduction of an approach to matching image segments in the context of an approximation space. The second step copes with the problem of estimating external camera parameters. The proposed solution to this problem uses 3D geometry rather than the fundamental matrix widely used in 2D to 3D conversion. In the proposed approach (DirectGS), the distances between reprojected rays for all image points are minimised. The contribution of the approach considered in this step is a definition of an optimal search space for solving the 2D to 3D conversion problem and introduction of an efficient algorithm that minimises reprojection error. In the third step, the problem of dense matching is considered. The contribution of this step is the introduction of a proposed approach to dense matching of 3D object structures that utilises the presence of points on lines in 3D space. The theory and experiments developed for this dissertation demonstrate the usefulness of the proposed system in the process of digitizing 3D information. The main advantage of the proposed approach is its low cost, simplicity in use for an untrained user and the high precision of reconstructed objects.

We propose algorithms for organization of images in wide-area sparse-view datasets. In such datasets, if the images overlap in scene content, they are related by wide-baseline geometric transformations. The challenge is to identify these relations even if the images sparingly overlap in their content. The images in a dataset are then grouped into sets of related images with the relations captured in each set as a basal (minimal and foundational) graph structures. Images form the vertices in the graph structure and the edges define the geometric relations between the images. We use these basal graphs for geometric walkthroughs and detection of noisy location (GPS) and orientation (magnetometer) information that may be stored with each image. We have five algorithmic contributions. First, we propose an algorithm BLOGS (Balanced Local and Global Search) that uses a novel hybrid Markov Chain Monte Carlo (MCMC) strategy called 'hop-diffusion' for epipolar geometry estimation between a pair of wide-baseline images that is 10 times faster and more accurate than the state-of-the-art. Hops are global searches and diffusions are local searches. BLOGS is able to handle very wide-baseline views characteristic of wide-area sparse-view datasets. It also produces a geometric match score between an image pair. Second, we propose a photometric match score, the Cumulative Correspondence Score (CCS). The proposed photometric scores are fast approximations of the computationally expensive geometric scores. Third, we use the photometric scores and the geometric scores to find groups of related images and to organize them in the form of basal graph structures using a novel hybrid algorithm we call theCOnnected component DIscovery by Minimally Specifying an Expensive Graph (CODIMSEG). The objective of the algorithm is to minimize the number of geometric estimations and yield results similar to what would be achieved if all-pair geometric matching were done. We compared the performances of the CCS and CODIMSEG algorithms with GIST (means summary of an image) and k-Nearest Neighbor (k-NN) based approaches. We found that CCS and CODIMSEG perform significantly better than GIST and k-NN respectively in identifying visually connected images. Our algorithm achieved more than 95% true positive rate at 0% false positive rate. Fourth, we propose a basal tree graph expansion algorithm to make the basal graphs denser for applications like geometric walk-throughs using the minimum Hamiltonian path algorithm and detection of noisy position (GPS) and orientation (magnetometer) tags. We propose two versions of geometric walkthroughs, one using minimum spanning tree based approximation of the minimum Hamiltonian path on the basal tree graphs and other using the Lin-Kernighan heuristic approximation on the expanded basal graph. Conversion of a non-linear tree structure to a linear path structure leads to discontinuities in path. The Lin-Kernighan algorithm on the expanded basal graphs is shown to be a better approach. Fifth, we propose a vision based geometric voting algorithm to detect noisy GPS and magnetometer tags using the basal graphs. This problem has never been addressed before to the best of our knowledge. We performed our experiments on the Nokia dataset (which has 243 images in the 'Lausanne' dataset and 105 images in the 'Demoset'), ArtQuad dataset (6514 images) and Oxford dataset (5063 images). All the three datasets are very different. Nokia dataset is a very wide-baseline sparse-view dataset. ArtQuad dataset is a wide-baseline dataset with denser views compared to the Nokia dataset. Both these datasets have GPS tagged images. Nokia dataset has magnetometer tags too. ArtQuad dataset has 348 images with the commercial GPS information as well as high precision differential GPS data which serves as ground truth for our noisy tag detection algorithm. Oxford dataset is a wide-baseline dataset with plenty of distracters that test the algorithm's capability to group images correctly. The larger datasets test the scalability of our algorithms. Visually inspected feature matches and image matches were used as ground truth in our experiments. All the experiments were done on a single PC.

This dissertation describes the design and implementation of a system that has been designed to extract 3D information from pairs of 2D images. System input consists of two images taken by an ordinary digital camera. System output is a full 3D model extracted from 2D images. There are no assumptions about the positions of the cameras during the time when the images are being taken, but the scene must not undergo any modifications. The process of extracting 3D information from 2D images consists of three basic steps. First, point matching is performed. The main contribution of this step is the introduction of an approach to matching image segments in the context of an approximation space. The second step copes with the problem of estimating external camera parameters. The proposed solution to this problem uses 3D geometry rather than the fundamental matrix widely used in 2D to 3D conversion. In the proposed approach (DirectGS), the distances between reprojected rays for all image points are minimised. The contribution of the approach considered in this step is a definition of an optimal search space for solving the 2D to 3D conversion problem and introduction of an efficient algorithm that minimises reprojection error. In the third step, the problem of dense matching is considered. The contribution of this step is the introduction of a proposed approach to dense matching of 3D object structures that utilises the presence of points on lines in 3D space. The theory and experiments developed for this dissertation demonstrate the usefulness of the proposed system in the process of digitizing 3D information. The main advantage of the proposed approach is its low cost, simplicity in use for an untrained user and the high precision of reconstructed objects.

A estimação de posição é o resultado direto da reconstrução de cenas, um dos ramos da visão computacional. É também uma informação importante para o controle de sistemas mecatrônicos, e em especial para os sistemas robóticos autônomos. Como uma aplicação de engenharia, o desempenho de tal sistema deve ser avaliado em termos de eficiência e eficácia, medidas traduzidas respectivamente pelo custo de processamento e pela quantificação do erro. A geometria epipolar é um campo da visão computacional que fornece formalismo matemático e técnicas de reconstrução de cenas a partir de uma par de imagens, através de pontos correspondentes entre elas. Através deste formalismo é possível determinar a incerteza dos métodos de estimação de posição, que são relativamente simples e podem atingir boa precisão. Dentre os sistemas robóticos autônomos destacam-se os ROVs - do inglês \"Remotely Operated Vehicles\" - ou veículos operados remotamente, muito utilizados em tarefas submarinas, e cuja necessidade crescente de autonomia motiva o desenvolvimento de um sensor de visão com características de baixo consumo de energia, flexibilidade e inteligência. Este sensor pode consistir de uma câmera CCD e algoritmos de reconstrução de cena baseados em geometria epipolar entre imagens. Este estudo visa fornecer um comparativo de resultados práticos da estimação de posição através da geometria epipolar entre imagens, como parte da implementação de um sensor de visão para robôs autônomos. Os conceitos teóricos abordados são: geometria projetiva, modelo de câmera, geometria epipolar, matriz fundamental, reconstrução projetiva, re-construção métrica, algoritmos de determinação da matriz fundamental, algoritmos de reconstrução métrica, incerteza da matriz fundamental e complexidade computacional. Os resultados práticos baseiam-se em simulações através de imagens geradas por computador e em montagens experimentais feitas em laboratório que simulam situações práticas. O processo de estimação de posição foi realizado através da implementação em MATLAB® 6.5 dos algoritmos apresentados na parte teórica, e os resultados comparados e analisados quanto ao erro e complexidade de execução. Dentre as principais conclusões é apresentado a melhor escolha para a implementação de sensor de visão de propósito geral - o Algoritmo de 8 Pontos Correspondentes Normalizado. São apresentadas também as condições de utilização de cada método e os cuidados necessários na interpretação dos resultados. / Position estimation is the direct result of scene reconstruction, one of computer vision\'s fields. It is also an important information for the control of mechanical systems - specially the autonomous robotic systems. As an engineering application, those systems\' performance must be evaluated in terms of efficiency and effectiveness, measured by processing costs and error quantification. The epipolar geometry is a field of computer vision that supply mathematical formalism and scene reconstruction techniques that are based on the correspondences between two images. Through this formalism it is possible to stipulate the uncertainty of the position estimation methods that are relatively simple and can give good accuracy. Among the autonomous robotic systems, the ROVs - Remotely Operated Vehicles - are of special interest, mostly employed in submarine activities, and whose crescent autonomy demand motivates the development of a vision sensor of low power consumption, flexibility and intelligence. This sensor may be constructed with a CCD camera and the scene reconstruction algorithms based on epipolar geometry. This work aims to build a comparison of practical results of position estimation through epipolar geometry, as part of a vision sensor implementation for autonomous robots. The theory presented in this work comprises of: projective geometry, camera model, epipolar geometry, fundamental matrix, projective reconstruction, metric reconstruction, fundamental matrix algorithms, metric reconstruction algorithms, fundamental matrix uncertainty, and computational complexity. The practical results are based on computer generated simulations and experimental assemblies that emulate practical issues. The position estimation was carried out by MATLAB® 6.5 implementations of the algorithms analyzed in the theoretical part, and the results are compared and analyzed in respect of the error and the execution complexity. The main conclusions are that the best algorithm choice for the implementation of a general purpose vision sensor is the Normalized 8 Point Algorithm, and the usage conditions of each method, besides the special considerations that must be observed at the interpretation of the results.

Sistemas de teleconferência e telepresença são ferramentas de comunicação cada vez mais comuns. Partindo da existência de um canal de comunicação de alta capacidade, busca-se permitir visualização tridimensional realista, sensível ao ponto de vista do usuário e que mantenha a estrutura física da cena sem conhecimento prévio de sua estrutura, por meio de câmeras estéreo. A partir de pares de imagens temporalmente coerentes são sintetizadas visões intermediárias da cena alvo, de modo que um usuário rastreado tenha a ilusão de estar vendo a cena real através de uma janela. O sistema implementado baseia-se em dispositivos que provejam melhor qualidade visual no componente de geração de imagens intermediárias e procura maximizar a facilidade de uso na parte de visualização, empregando dispositivos domésticos e dispensando procedimentos complexos de calibração. / Teleconference and telepresence systems are increasingly common. Based on the existence of a high capacity communication channel, it is aimed to provide user\'s point of view sensible realistic 3D visualization without physical distortion or any prior knowledge about the structure of the scene, using stereo cameras. Pairs of temporally coherent images are used to generate intermediary view of the target scene so that a tracked user will have the illusion of posing in front so the real scene. The implemented system is based on high visual quality components on the side of views generation and tries to maximize ease of use in the visualization part by using commodity components and being free so complex calibration procedures.