In recent years there has been significant progress in increasing the scope, accuracy and flexibility of 3D photography methods. However there are still significant open problems where complex optical properties of mirroring or transparent objects cause many assumptions of traditional algorithms to break down. In this work we present three approaches that attempt to deal with some of these challenges using a few camera views and simple illumination. First, we consider the problem of reconstructing the 3D position and surface normal of points on a time-varying refractive surface. We show that two viewpoints are sufficient to solve this problem in the general case, even if the refractive index is unknown. We introduce a novel ``stereo matching'' criterion called refractive disparity, appropriate for refractive scenes, and develop an optimization-based algorithm for individually reconstructing the position and normal of each point projecting to a pixel in the input views. Second, we present a new method for reconstructing the exterior surface of a complex transparent scene with inhomogeneous interior. We capture images from each viewpoint while moving a proximal light source to a 2D or 3D set of positions giving a 2D (or 3D) dataset per pixel, called the scatter-trace. The key is that while light transport within a transparent scene's interior can be exceedingly complex, a pixel's scatter trace has a highly-constrained geometry that reveals the direct surface reflection, and leads to a simple ``Scatter-trace stereo'' algorithm for computing the exterior surface geometry. Finally, we develop a reconstruction system for scenes with reflectance properties ranging from diffuse to specular. We capture images of the scene as it is illuminated by a planar, spatially non-uniform light source. Then we show that if the source is translated to a parallel position away from the scene, a particular scene point integrates a magnified region of light from the plane. We observe this magnification at each pixel and show how it relates to the source-relative depth of the surface. Next we show how calibration relating the camera and source planes allows for robustness to specular objects and recovery of 3D surface points.

Σκοπός της διπλωματικής εργασίας είναι να παρουσιαστούν μερικές από τις πιο αξιοσημείωτες τεχνικές για τρισδιάστατη αναπαράσταση χώρου από βιντεοσκοπήσεις κινούμενου αυτοκινήτου. Αρχικά, το πρώτο κομμάτι είναι προσανατολισμένο στην περιγραφή των εφαρμογών που ακολουθεί η τρισδιάστατη αναπαράσταση χώρου από βιντεοσκοπήσεις και επίσης γίνεται εισαγωγή στις αρχικές έννοιες και στα βασικά μαθηματικά που χρησιμοποιούνται. Έπειτα περιγράφεται μια μέθοδος τρισδιάστατης αναπαράστασης από βιντεοσκόπηση με την χρήση απλών καμερών. Ο αλγόριθμος που περιγράφεται εδώ χρησιμοποιεί τις εικόνες από την απλή κάμερα και απαιτεί επιπλέον ότι κάθε δείκτης στο αντικείμενο συνδέεται με τουλάχιστον έναν άλλο δείκτη από σωστή απόσταση. Στο τρίτο κομμάτι της εργασίας περιγράφεται μια ακόμη μέθοδος η οποία στηρίζεται σε γραμμικά κινούμενα αντικείμενα. / The goal of this bachelor thesis is to present some of the most remarkable techniques of the three-dimensional reconstruction site from video animation moving car. At first, the first piece is oriented at the description of the application used by the three-dimensional reconstruction site of video and it is also an introduction to the initial concept and to the basic mathematics that are used. After that, a method of 3D video reconstruction using simple cameras is being described. The algorithm being described here, uses the images from the simple camera and requires that in addition that every object indication links with at least one other indicator from a known distance. At the third part one more method is described that is founded in linear moving objects.

Thesis (MScEng (Mathematical Sciences. Applied Mathematics))--Stellenbosch University, 2008. / When building or improving a petrochemical plant, drawings are used extensively in the design process. However, existing petrochemical plants seldom match their drawings, or the drawings are lost, forcing the need to generate a 3D model of the structure of the plant. In this thesis photogrammetry is investigated as a method of generating a digital 3D model of an existing plant. Camera modeling, target extraction and 3D reconstruction are discussed in detail, and a real-world system is investigated.

As escavações subterrâneas têm estado sempre afetadas pelas condições dos maciços, os quais apresentam diferentes características e comportamentos frente a deformações ocasionadas após a escavação de uma galeria ou túnel. Em virtude disto, torna-se necessário uma análise dos parâmetros que permitam estabelecer as condições de segurança do túnel e propõe-se o uso de um sistema estéreo de rastreamento ótico para analisar o deslocamento de artefatos (conjunto de marcadores) instalados sobre o contorno das faces do túnel, a fim de detectarem-se deslocamentos do maciço em pontos estabelecidos com precisão menor a ±1 mm. O sistema proposto opera na região espectral do infravermelho, permitindo assim uma operação em ambientes com baixa luminosidade e com inserção de ruído ótico. Adicionalmente o trabalho inclui resultados de validação e desenvolvimento do sistema de visão estéreo baixo condições de dois cenários de diferente volume espacial, assim como o procedimento realizado para a calibração dos parâmetros intrínsecos e extrínsecos dos sensores do sistema (câmeras monocromáticas). Os resultados obtidos apresentam uma precisão de ±0,1881 mm para artefatos com três marcadores e uma precisão de ±0,4952 para artefatos com quatro marcadores. Finalmente, o sistema opera com uma sensibilidade de 0,99 mm na determinação da posição de um artefato associado a um ponto do perfil do túnel, e desta forma, podem-se definir as condições de estabilidade do maciço rochoso e os requisitos para auxilio as decisões de métodos de tratamento e aumento da segurança. / Underground excavations have always been affected by the conditions of mass, which have different characteristics and behaviors against deformation caused after excavation of a gallery or tunnel. Because of this, it is necessary an analysis of the parameters needed to establish the safety conditions of the tunnel and it is proposed the use of a stereo optical tracking to analyze the displacement of artifacts (set of markers) installed on the contour of tunnels’ faces in order to detect displacements of the mass points established with a precision lower than ±1 mm. This system operates in the infrared spectral region to take advantage of the low light and reduce the optical noise in environments with grace. Further work includes validation results and development of low stereo vision conditions of two different scenarios spatial volume, as well as the procedure carried out to calibrate the intrinsic and extrinsic parameters of the system sensors (monochrome camera) system. The results show an accuracy of ± 0,1881 mm for artifacts with three markers and an accuracy of ± 0,4952 for artifacts with four markers. Finally, the system operates with a sensitivity of 0,99 mm in the determination of an artifact associated with a point in the tunnel profile position, and thus, one can define the conditions for stability of the rock mass and assistance requirements for the decisions of treatment methods and increased.

Ce travail de thèse présente un nouveau modèle de projection pour les cameras fisheye, qui est mathématiquement simple et pourtant montre une haute précision une fois appliqué aux caméras réelles. Les propriétés géométriques de ce modèle ont été analysées en utilisant le concept de surface de projection, introduit dans ce travail. En particulier, une inverse analytique de ce modèle a été établie, ainsi qu'une équation implicite des projections de lignes droites. Cette dernière nous a permis de développer une méthode de reconstruction 3D directe basée vision pour les caméras fisheye sans rectifier les images. Cela a été fait grâce à une algorithme de rastérisation de courbes implicites. Cet algorithme de reconstruction nous permet d'employer le Semi-Global Matching pour obtenir une reconstruction 3D précise. Tous ces éléments ont été employés dans un système de localisation visuelle directe avec deux méthodes de recalage d'images : minimisation d'erreur photométrique et maximisation d'information mutuelle. L'étalonnage intrinsèque et extrinsèque d'un robot mobile équipé de caméras fisheye a été considéré et une toolbox d'étalonnage a été développée. / This thesis presents a novel projection model for fisheye cameras, which is mathematically simple and yet shows a high precision when applied to real cameras. Geometric properties of the model have been analyzed using the concept of projection surface, introduced in this work. In particular, a closed-form inverse mapping and an implicit equation for straight line projection have been found. This fact has been used to develop a method of direct stereo correspondence on raw fisheye images via rasterization of implicit curve. This correspondence algorithm allows us to apply the Semi-Global Matching algorithm to get an accurate 3D reconstruction using fisheye stereo systems. All these elements have been shown to be applicable to a direct visual localization system with two different methods of image registration: direct photometric error minimization and mutual information maximization. Intrinsic and extrinsic calibration of a mobile robot with fisheye cameras has been considered and a toolbox for such a calibration has been developed.

In this thesis, a novel and fast method to compute the dense disparity map of a stereo pair of images is presented. Most of the current stereo matching algorithms are ill suited for real-time matching owing to their time complexity. Methods that concentrate on providing a real-time performance, sacrifice much in accuracy. The presented method, Fast Estimation of Dense Disparity Map Using Pivot Points (FEDDUP), uses a hierarchical approach towards reduction of search space to find the correspondences. The hierarchy starts with a set of points and then it moves on to a mesh with which the edge pixels are matched. This results in a semi-global disparity map. The semi global disparity map is then used as a soft constraint to find the correspondences of the remaining points. This process delivers good real-time performance with promising accuracy.

Componentes metálicos em operação podem estar sujeitos a diversas condições de operação deletérias. Visando avaliar os riscos de operação para evitar falhas, a análise de integridade estrutural é uma ferramenta amplamente aplicada e requer constante evolução. Por utilizar dados de ensaios não destrutivos, essa ferramenta requer cada vez mais precisão, para aprimorar seus resultados e reduzir ao máximo as falhas de componentes em operação. Por apresentar menor tempo de inspeção e maior probabilidade de detecção, o Phased Array surge como alternativa às técnicas convencionais de ultrassom. Dentre as técnicas de Phased Array, o Total Focusing Method (TFM) apresenta um dos resultados mais promissores, apresentando grande vantagem sobre técnicas convencionais de ultrassom e de Phased Array devido a sua melhor precisão e fácil interpretação dos resultados. Neste trabalho, o TFM foi aplicado em diversos blocos contendo descontinuidades usinadas, as quais simulam diferentes tipos de defeitos. O foco do TFM foi gerar blocos em três dimensões (3D) dos defeitos, facilitando a interpretação dos resultados, além da possibilidade da inserção desses blocos em softwares de análise de integridade estrutural, melhorando a precisão dos resultados. O TFM foi aplicado através do tratamento de dados de um sensor linear de Phased Array, com frequência de 5 MHz e com 64 elementos ativos. Nesse trabalho, foram desenvolvidas e aplicadas novas etapas no algoritmo do TFM para melhorar a precisão dos resultados, como a compensação da perda de energia e o half-skip. As etapas adicionais aplicadas ao algoritmo do TFM geraram bons resultados para entalhe e furos usinados nos blocos de aço carbono. Além dos blocos com defeitos usinados, foi realizada a inspeção de um bloco de aço inoxidável austenítico soldado. Para o bloco soldado, os resultados não foram satisfatórios, não sendo possível detectar defeitos existentes. Após a aplicação do TFM, foi simulado o deslocamento do sensor ao longo da peça, gerando diversas imagens 2D, as quais foram conectadas via isosuperfícies, gerando sólidos 3D dos defeitos presentes em cada bloco. Por fim, esses blocos foram exportados para um software CAD, apresentando excelente correspondência. / Metallic components in operation can be submitted to several harming operation conditions. Aiming to evaluate the operational risks to avoid failure, the structural integrity analysis is a widely applied tool and requires constant evolution. As it uses non-destructive testing data, this tool requires increasingly precision, to enhance its results and reduce the majority of the operating components failures. Because of the shorter inspection time and greater probability of detection, Phased Array is an alternative to conventional ultrasonic techniques. Among the techniques of Phased Array, the Total Focusing Method (TFM) presents one of the most promising results, offering great advantage over conventional ultrasonic and Phased Array techniques due to its better precision and easy interpretation of results. In this work, the TFM was applied to many blocks, which contain machined discontinuities simulating several defects. The objective of applying the TFM was to generate three-dimensional (3D) blocks, turning the interpretation of the results easier and giving the possibility to insert these blocks into a structural integrity analysis software, enhancing the results precision. The TFM algorithm was applied through treatment of data from a 5 MHz linear Phased Array sensor with 64 active elements. In this work, it were developed and applied new steps in the TFM algorithm, as energy loss compensation and half-skip interaction, to improve its precision. The additional steps applied to the TFM algorithm achieved great results for notch and holes machined in carbon steel blocks. In addition to the blocks with machined defects, a welded austenitic stainless steel block was inspected. For the welded block, the results were not satisfactory and it was not possible to detect the present defects. After the application of the TFM, the displacement of the transducer along the block was simulated, generating several 2D images, which were connected via isosurfaces, generating 3D solids corresponding to the defects present in each block. To conclude, these blocks were exported to a CAD software, presenting excellent matching.

As escavações subterrâneas têm estado sempre afetadas pelas condições dos maciços, os quais apresentam diferentes características e comportamentos frente a deformações ocasionadas após a escavação de uma galeria ou túnel. Em virtude disto, torna-se necessário uma análise dos parâmetros que permitam estabelecer as condições de segurança do túnel e propõe-se o uso de um sistema estéreo de rastreamento ótico para analisar o deslocamento de artefatos (conjunto de marcadores) instalados sobre o contorno das faces do túnel, a fim de detectarem-se deslocamentos do maciço em pontos estabelecidos com precisão menor a ±1 mm. O sistema proposto opera na região espectral do infravermelho, permitindo assim uma operação em ambientes com baixa luminosidade e com inserção de ruído ótico. Adicionalmente o trabalho inclui resultados de validação e desenvolvimento do sistema de visão estéreo baixo condições de dois cenários de diferente volume espacial, assim como o procedimento realizado para a calibração dos parâmetros intrínsecos e extrínsecos dos sensores do sistema (câmeras monocromáticas). Os resultados obtidos apresentam uma precisão de ±0,1881 mm para artefatos com três marcadores e uma precisão de ±0,4952 para artefatos com quatro marcadores. Finalmente, o sistema opera com uma sensibilidade de 0,99 mm na determinação da posição de um artefato associado a um ponto do perfil do túnel, e desta forma, podem-se definir as condições de estabilidade do maciço rochoso e os requisitos para auxilio as decisões de métodos de tratamento e aumento da segurança. / Underground excavations have always been affected by the conditions of mass, which have different characteristics and behaviors against deformation caused after excavation of a gallery or tunnel. Because of this, it is necessary an analysis of the parameters needed to establish the safety conditions of the tunnel and it is proposed the use of a stereo optical tracking to analyze the displacement of artifacts (set of markers) installed on the contour of tunnels’ faces in order to detect displacements of the mass points established with a precision lower than ±1 mm. This system operates in the infrared spectral region to take advantage of the low light and reduce the optical noise in environments with grace. Further work includes validation results and development of low stereo vision conditions of two different scenarios spatial volume, as well as the procedure carried out to calibrate the intrinsic and extrinsic parameters of the system sensors (monochrome camera) system. The results show an accuracy of ± 0,1881 mm for artifacts with three markers and an accuracy of ± 0,4952 for artifacts with four markers. Finally, the system operates with a sensitivity of 0,99 mm in the determination of an artifact associated with a point in the tunnel profile position, and thus, one can define the conditions for stability of the rock mass and assistance requirements for the decisions of treatment methods and increased.

Componentes metálicos em operação podem estar sujeitos a diversas condições de operação deletérias. Visando avaliar os riscos de operação para evitar falhas, a análise de integridade estrutural é uma ferramenta amplamente aplicada e requer constante evolução. Por utilizar dados de ensaios não destrutivos, essa ferramenta requer cada vez mais precisão, para aprimorar seus resultados e reduzir ao máximo as falhas de componentes em operação. Por apresentar menor tempo de inspeção e maior probabilidade de detecção, o Phased Array surge como alternativa às técnicas convencionais de ultrassom. Dentre as técnicas de Phased Array, o Total Focusing Method (TFM) apresenta um dos resultados mais promissores, apresentando grande vantagem sobre técnicas convencionais de ultrassom e de Phased Array devido a sua melhor precisão e fácil interpretação dos resultados. Neste trabalho, o TFM foi aplicado em diversos blocos contendo descontinuidades usinadas, as quais simulam diferentes tipos de defeitos. O foco do TFM foi gerar blocos em três dimensões (3D) dos defeitos, facilitando a interpretação dos resultados, além da possibilidade da inserção desses blocos em softwares de análise de integridade estrutural, melhorando a precisão dos resultados. O TFM foi aplicado através do tratamento de dados de um sensor linear de Phased Array, com frequência de 5 MHz e com 64 elementos ativos. Nesse trabalho, foram desenvolvidas e aplicadas novas etapas no algoritmo do TFM para melhorar a precisão dos resultados, como a compensação da perda de energia e o half-skip. As etapas adicionais aplicadas ao algoritmo do TFM geraram bons resultados para entalhe e furos usinados nos blocos de aço carbono. Além dos blocos com defeitos usinados, foi realizada a inspeção de um bloco de aço inoxidável austenítico soldado. Para o bloco soldado, os resultados não foram satisfatórios, não sendo possível detectar defeitos existentes. Após a aplicação do TFM, foi simulado o deslocamento do sensor ao longo da peça, gerando diversas imagens 2D, as quais foram conectadas via isosuperfícies, gerando sólidos 3D dos defeitos presentes em cada bloco. Por fim, esses blocos foram exportados para um software CAD, apresentando excelente correspondência. / Metallic components in operation can be submitted to several harming operation conditions. Aiming to evaluate the operational risks to avoid failure, the structural integrity analysis is a widely applied tool and requires constant evolution. As it uses non-destructive testing data, this tool requires increasingly precision, to enhance its results and reduce the majority of the operating components failures. Because of the shorter inspection time and greater probability of detection, Phased Array is an alternative to conventional ultrasonic techniques. Among the techniques of Phased Array, the Total Focusing Method (TFM) presents one of the most promising results, offering great advantage over conventional ultrasonic and Phased Array techniques due to its better precision and easy interpretation of results. In this work, the TFM was applied to many blocks, which contain machined discontinuities simulating several defects. The objective of applying the TFM was to generate three-dimensional (3D) blocks, turning the interpretation of the results easier and giving the possibility to insert these blocks into a structural integrity analysis software, enhancing the results precision. The TFM algorithm was applied through treatment of data from a 5 MHz linear Phased Array sensor with 64 active elements. In this work, it were developed and applied new steps in the TFM algorithm, as energy loss compensation and half-skip interaction, to improve its precision. The additional steps applied to the TFM algorithm achieved great results for notch and holes machined in carbon steel blocks. In addition to the blocks with machined defects, a welded austenitic stainless steel block was inspected. For the welded block, the results were not satisfactory and it was not possible to detect the present defects. After the application of the TFM, the displacement of the transducer along the block was simulated, generating several 2D images, which were connected via isosurfaces, generating 3D solids corresponding to the defects present in each block. To conclude, these blocks were exported to a CAD software, presenting excellent matching.

Les tâches réalisées en traitement d'image tendent à devenir de plus en plus complexes. Par exemple, dans le contexte routier, les systèmes d'aide à la conduite, (Advanced driver-assistance systems), visent à une automatisation complète de la tâche de conduite. L’évaluation de la fiabilité représente un enjeu important pour ce type d’application. Face à la difficulté des tâches à réaliser, les chaînes de traitements sont souvent divisées en de nombreuses étapes de calculs de sorte qu'il est difficile de caractériser les sorties de la chaîne en fonction des perturbations des entrées. Les étapes du traitement consistent le plus souvent en des problèmes formulés comme la minimisation d'une énergie. Cette énergie est généralement difficile à optimiser, ce qui nécessite la mise en œuvre de méthodes d’optimisation adaptées. Dans cette thèse, nous cherchons à caractériser la solution d’un traitement à partir des calculs réalisés au cours de l’étape d'optimisation. Cette approche nous a permis de proposer des indices de stabilité de la solution dans le cadre de deux méthodes d’optimisation discrètes : la coupure de graphe et la programmation dynamique. Tout d’abord, nous nous sommes intéressés au problème de la reconstruction stéréoscopique en contexte routier et au dé-bruitage, dans le cadre de l’optimisation par coupure de graphe. Les modèles issus de l’interprétation bayésienne amènent à optimiser des énergies qui ne peuvent pas être traitées avec les schémas d’optimisation classiques par fusion binaire. Nous avons proposé un schéma adapté qui met en jeu des fusions binaires par expansion et par saut. L’application de ce schéma aux problèmes de la reconstruction stéréoscopique et au dé-bruitage, nous a permis d’obtenir des solutions possédant les caractéristiques que nous recherchions : des contours d’objets nets et des dégradés progressifs dans les zones homogènes. Ensuite, dans le contexte de la programmation dynamique, nous avons réinterprété l’a priori mis en jeu dans la méthode de reconstruction Semi-Global Matching ainsi que certaines de ses variantes. Nous avons proposé d’ajouter un paramètre à ces méthodes afin de modifier les directions privilégiées par l’a priori. Enfin, nous avons proposé des indices de stabilité de la solution dans le cadre de la coupure de graphe et de la programmation dynamique. La prise en compte de ces indices, dans une étape de raffinement des solutions, permet une amélioration des résultats / Problems solved by image processing tend to be more and more complex. For instance, in road context, ADAS (Advanced driver-assistance systems) aim to a completely automatic diving tack. Evaluating system reliability is an important challenge in that case. These tasks being hard to perform, processing chains are often divide in numerous processing steps. As a consequence, characterizing the output using the input of the chain is not obvious. Most of the time, image processing steps are formulate as an energy minimization. These energies are often hard to minimize and need to apply suitable optimization methods. In this thesis, we aim to characterize the solution during the optimization step. Using this approach, we proposed stability index with two discrete optimization methods : graph-cut and dynamic optimization. First, we focused on stereoscopic reconstruction problem in road context and on denoising problem using graph-cut. Models obtained by Bayesian interpretation lead to optimize energies witch cannot be handled by classical binary fusion optimization scheme. We proposed a suitable scheme composed of fusion by expansion and fusions by step. When this scheme is apply to stereoscopic reconstruction and denoising, obtained solution have the wanted characteristics : sharp edges and shading in homogeneous areas. Next, in dynamic programming context, we reinterpreted the prior used in Semi-Global Matching (SGM) stereoscopic reconstruction method and in some of its variants. We proposed an additional parameter in order to modify the favored direction in the prior. At last, we proposed stability index of the solution in graph-cut and dynamic programing context. Using this index in a solution refinement step shows improvements