A obtenção e a reconstrução da geometria tridimensional (3D) de objetos e ambientes têm importância crescente em áreas como visão computacional e computação gráfica. As formas atuais de obtenção e reconstrução 3D necessitam de equipamentos e montagens sofisticadas que, por conseqüência, têm custos elevados e aplicação limitada. Este trabalho apresenta criticamente os principais algoritmos para a reconstrução 3D a partir de par de imagens estéreo e identifica os mais viáveis para utilização com equipamentos convencionais. Por meio da implementação de alguns destes algoritmos, da comparação dos resultados obtidos em sua execução e também pela comparação com os resultados encontrados na literatura, são identificadas as principais deficiências. São propostas adequações aos algoritmos existentes, em particular, é apresentada a proposta da técnica das faixas que proporciona a redução drástica no consumo de memória para o processamento da geometria 3D e que possui desempenho computacional melhor em relação às técnicas tradicionais. Foi implementado um protótipo de sistema de reconstrução 3D que permite a reconstrução pelas diferentes técnicas estudadas e propostas, bem como permite visualizar o cenário reconstruído sob diferentes pontos de vista de forma interativa. / The acquisition and reconstruction of three-dimensional (3D) geometry of objects and environments have their importance growing in areas such as Computer Vision and Computer Graphics. The current methods to acquire and reconstruct three-dimensional data need sophisticated equipments and assemblies, which have expensive costs and limited applications. This work presents the main algorithms for 3D reconstruction using a pair of stereo images and identifies which are viable to use with conventional equipments. Through the implementation of some of these algorithms, by comparing the results obtained and comparing with the results presented in the literature, the main limitations were identified. This work proposes adjustments in the existing algorithms, in particular it proposes the stripping technique, which provides a huge memory usage reduction for 3D geometry processing and better computing performance if compared with traditional approaches. A prototype system for 3D reconstruction was implemented, which allows the reconstruction using the different researched and proposed techniques and allows interactive visualization of the reconstructed scene in different angles.

A obtenção e a reconstrução da geometria tridimensional (3D) de objetos e ambientes têm importância crescente em áreas como visão computacional e computação gráfica. As formas atuais de obtenção e reconstrução 3D necessitam de equipamentos e montagens sofisticadas que, por conseqüência, têm custos elevados e aplicação limitada. Este trabalho apresenta criticamente os principais algoritmos para a reconstrução 3D a partir de par de imagens estéreo e identifica os mais viáveis para utilização com equipamentos convencionais. Por meio da implementação de alguns destes algoritmos, da comparação dos resultados obtidos em sua execução e também pela comparação com os resultados encontrados na literatura, são identificadas as principais deficiências. São propostas adequações aos algoritmos existentes, em particular, é apresentada a proposta da técnica das faixas que proporciona a redução drástica no consumo de memória para o processamento da geometria 3D e que possui desempenho computacional melhor em relação às técnicas tradicionais. Foi implementado um protótipo de sistema de reconstrução 3D que permite a reconstrução pelas diferentes técnicas estudadas e propostas, bem como permite visualizar o cenário reconstruído sob diferentes pontos de vista de forma interativa. / The acquisition and reconstruction of three-dimensional (3D) geometry of objects and environments have their importance growing in areas such as Computer Vision and Computer Graphics. The current methods to acquire and reconstruct three-dimensional data need sophisticated equipments and assemblies, which have expensive costs and limited applications. This work presents the main algorithms for 3D reconstruction using a pair of stereo images and identifies which are viable to use with conventional equipments. Through the implementation of some of these algorithms, by comparing the results obtained and comparing with the results presented in the literature, the main limitations were identified. This work proposes adjustments in the existing algorithms, in particular it proposes the stripping technique, which provides a huge memory usage reduction for 3D geometry processing and better computing performance if compared with traditional approaches. A prototype system for 3D reconstruction was implemented, which allows the reconstruction using the different researched and proposed techniques and allows interactive visualization of the reconstructed scene in different angles.

A novel and efficient methodology is presented for the conversion of stereo to multi-view video in order to address the 3D content requirements for the next generation 3D-TVs and auto-stereoscopic multi-view displays. There are two main algorithmic blocks in such a conversion system / stereo matching and virtual view rendering that enable extraction of 3D information from stereo video and synthesis of inexistent virtual views, respectively. In the intermediate steps of these functional blocks, a novel edge-preserving filter is proposed that recursively constructs connected support regions for each pixel among color-wise similar neighboring pixels. The proposed recursive update structure eliminates pre-defined window dependency of the conventional approaches, providing complete content adaptibility with quite low computational complexity. Based on extensive tests, it is observed that the proposed filtering technique yields better or competitive results against some leading techniques in the literature. The proposed filter is mainly applied for stereo matching to aggregate cost functions and also handles occlusions that enable high quality disparity maps for the stereo pairs. Similar to box filter paradigm, this novel technique yields matching of arbitrary-shaped regions in constant time. Based on Middlebury benchmarking, the proposed technique is currently the best local matching technique in the literature in terms of both precision and complexity. Next, virtual view synthesis is conducted through depth image based rendering, in which reference color views of left and right pairs are warped to the desired virtual view using the estimated disparity maps. A feedback mechanism based on disparity error is introduced at this step to remove salient distortions for the sake of visual quality. Furthermore, the proposed edge-aware filter is re-utilized to assign proper texture for holes and occluded regions during view synthesis. Efficiency of the proposed scheme is validated by the real-time implementation on a special graphics card that enables parallel computing. Based on extensive experiments on stereo matching and virtual view rendering, proposed method yields fast execution, low memory requirement and high quality outputs with superior performance compared to most of the state-of-the-art techniques.

The polarization of light was introduced last ten years ago in the field of imaging system is a physical phenomenon that can be controlled for the purposes of the vision system. As that found in the human eyes, in general the imaging sensors are not under construction which is sensitive to the polarization of light. These properties can be measured by adding optical components on a conventional camera. The purpose of this thesis is to develop an imaging system that is sensitive both to the stereoscopic and to the state of polarization. As well as the visual system on a various of insects in nature such as bees, that are have capability to move in space by extracted relevant information from the polarization. The developed prototype should be possible to reconstruct threedimensional of points of interest with the issues associated with a set of parameters of the state of polarization. The proposed system consists of two cameras, each camera equipped with liquid crystal components to obtain two images with different directions of polarization. For each acquisition, four images are acquired: two for each camera. Raised by the key of main capability to return polarization information from two different cameras. After an initial calibration step; geometric and photometric, the mapping of points of interest process is made difficult because of the optical components placed in front of different lenses. A detailed study of different methods of mapping was used to select sensitivity to the polarization effects. Once points are mapped, the polarization parameters of each point are calculated from the four values from four images acquired. The results on real scenes show the feasibility and desirability of this imaging system for robotic applications.

Made available in DSpace on 2014-12-17T14:55:44Z (GMT). No. of bitstreams: 1 AllanDGA_DISSERT.pdf: 1992696 bytes, checksum: 87d8b1dbc6fe4df6df2f85f90481f9be (MD5) Previous issue date: 2010-01-13 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior / This study aims to seek a more viable alternative for the calculation of differences in images of stereo vision, using a factor that reduces heel the amount of points that are considered on the captured image, and a network neural-based radial basis functions to interpolate the results. The objective to be achieved is to produce an approximate picture of disparities using algorithms with low computational cost, unlike the classical algorithms / O presente trabalho visa buscar uma alternativa mais vi?vel para o c?lculo das disparidades em imagens de vis?o est?reo, utilizando um fator de salto que reduz a quantidade de pontos que s?o considerados da imagem capturada, e uma rede neural baseada em fun??es de base radial para interpolar os resultados obtidos. O objetivo a ser alcan?ado ? produzir uma imagem de disparidades aproximada da real com algoritmos de baixo custo computacional, diferentemente dos algoritmos tradicionais

Three-dimensional information is often used in robotics and 3D-mapping. There exist several ways to obtain a three-dimensional map. However, the time of flight used in the laser scanners or the structured light utilized by Kinect-like sensors sometimes are not sufficient. In this thesis, we investigate two CNN based stereo matching methods for obtaining 3D-information from a grayscaled pair of rectified images.While the state-of-the-art stereo matching method utilize a Siamese architecture, in this project a two-channel and a two stream network are trained in an attempt to outperform the state-of-the-art. A set of experiments were performed to achieve optimal hyperparameters. By changing one parameter at the time, the networks with architectures mentioned above are trained. After a completed training the networks are evaluated with two criteria, the error rate, and the runtime.Due to time limitations, we were not able to find optimal learning parameters. However, by using settings from [17] we train a two-channel network that performed almost on the same level as the state-of-the-art. The error rate on the test data for our best architecture is 2.64% while the error rate for the state-of-the-art Siamese network is 2.62%. We were not able to achieve better performance than the state-of-the-art, but we believe that it is possible to reduce the error rate further. On the other hand, the state-of-the-art Siamese stereo matching network is more efficient and faster during the disparity estimation. Therefore, if the time efficiency is prioritized, the Siamese based network should be considered.

Recent years have witnessed tremendous success in using deep learning approaches to handle low-level vision problems. Most of the deep learning based methods address the low-level vision problem by training a neural network to approximate the mapping from the inputs to the desired ground truths. However, directly learning this mapping is usually difficult and cannot achieve ideal performance. Besides, under the setting of unsupervised learning, the general deep learning approach cannot be used. In this thesis, we investigate and address several problems in low-level vision using the proposed approaches. To learn a better mapping using the existing data, an indirect domain shift mechanism is proposed to add explicit constraints inside the neural network for single image dehazing. This allows the neural network to be optimized across several identified neighbours, resulting in a better performance. Despite the success of the proposed approaches in learning an improved mapping from the inputs to the targets, three problems of unsupervised learning is also investigated. For unsupervised monocular depth estimation, a teacher-student network is introduced to strategically integrate both supervised and unsupervised learning benefits. The teacher network is formed by learning under the binocular depth estimation setting, and the student network is constructed as the primary network for monocular depth estimation. In observing that the performance of the teacher network is far better than that of the student network, a knowledge distillation approach is proposed to help improve the mapping learned by the student. For single image dehazing, the current network cannot handle different types of haze patterns as it is trained on a particular dataset. The problem is formulated as a multi-domain dehazing problem. To address this issue, a test-time training approach is proposed to leverage a helper network in assisting the dehazing network adapting to a particular domain using self-supervision. In lossy compression system, the target distribution can be different from that of the source and ground truths are not available for reference. Thus, the objective is to transform the source to target under a rate constraint, which generalizes the optimal transport. To address this problem, theoretical analyses on the trade-off between compression rate and minimal achievable distortion are studied under the cases with and without common randomness. A deep learning approach is also developed using our theoretical results for addressing super-resolution and denoising tasks. Extensive experiments and analyses have been conducted to prove the effectiveness of the proposed deep learning based methods in handling the problems in low-level vision. / Thesis / Doctor of Philosophy (PhD)

<p>Many different approaches have been taken towards solving the stereo correspondence problem and great progress has been made within the field during the last decade. This is mainly thanks to newly evolved global optimization techniques and better ways to compute pixel dissimilarity between views. The most successful algorithms are based on approaches that explicitly model smoothness assumptions made about the physical world, with image segmentation and plane fitting being two frequently used techniques.</p><p>Within the project, a survey of state of the art stereo algorithms was conducted and the theory behind them is explained. Techniques found interesting were implemented for experimental trials and an algorithm aiming to achieve state of the art performance was implemented and evaluated. For several cases, state of the art performance was reached.</p><p>To keep down the computational complexity, an algorithm relying on local winner-take-all optimization, image segmentation and plane fitting was compared against minimizing a global energy function formulated on pixel level. Experiments show that the local approach in several cases can match the global approach, but that problems sometimes arise – especially when large areas that lack texture are present. Such problematic areas are better handled by the explicit modeling of smoothness in global energy minimization.</p><p>Lastly, disparity estimation for image sequences was explored and some ideas on how to use temporal information were implemented and tried. The ideas mainly relied on motion detection to determine parts that are static in a sequence of frames. Stereo correspondence for sequences is a rather new research field, and there is still a lot of work to be made.</p>

Image registration is a fundamental problem that can be found in a diverse range of fields within the research community. It is used in areas such as engineering, science, medicine, robotics, computer vision and image processing, which often require the process of developing a spatial mapping between sets of data. Registration plays a crucial role in the medical imaging field where continual advances in imaging modalities, including MRI, CT and PET, allow the generation of 3D images that explicitly outline detailed in vivo information of not only human anatomy, but also human function. Mutual Information (MI) is a popular entropy-based similarity measure which has found use in a large number of image registration applications. Stemming from information theory, this measure generally outperforms most other intensity-based measures in multimodal applications as it does not assume the existence of any specific relationship between image intensities. It only assumes a statistical dependence. The basic concept behind any approach using MI is to find a transformation, which when applied to an image, will maximise the MI between two images. This thesis presents research using MI in three major topics encompassed by the computer vision and medical imaging field: rigid image registration, stereo vision, and non-rigid image registration. In the rigid domain, a novel gradient-based registration algorithm (MIGH) is proposed that uses Parzen windows to estimate image density functions and Gauss-Hermite quadrature to estimate the image entropies. The use of this quadrature technique provides an effective and efficient way of estimating entropy while bypassing the need to draw a second sample of image intensities (a procedure required in previous Parzen-based MI registration approaches). It is possible to achieve identical results with the MIGH algorithm when compared to current state of the art MI-based techniques. These results are achieved using half the previously required sample sizes, thus doubling the statistical power of the registration algorithm. Furthermore, the MIGH technique improves algorithm complexity by up to an order of N, where N represents the number of samples extracted from the images. In stereo vision, a popular passive method of depth perception, new extensions have been pro- posed in order to increase the robustness of MI-based stereo matching algorithms. Firstly, prior probabilities are incorporated into the MI measure to considerably increase the statistical power of the matching windows. The statistical power, directly related to the number of samples, can become too low when small matching windows are utilised. These priors, which are calculated from the global joint histogram, are tuned to a two level hierarchical approach. A 2D match surface, in which the match score is computed for every possible combination of template and matching windows, is also utilised to enforce left-right consistency and uniqueness constraints. These additions to MI-based stereo matching significantly enhance the algorithms ability to detect correct matches while decreasing computation time and improving the accuracy, particularly when matching across multi-spectra stereo pairs. MI has also recently found use in the non-rigid domain due to a need to compute multimodal non-rigid transformations. The viscous fluid algorithm is perhaps the best method for re- covering large local mis-registrations between two images. However, this model can only be used on images from the same modality as it assumes similar intensity values between images. Consequently, a hybrid MI-Fluid algorithm is proposed to compute a multimodal non-rigid registration technique. MI is incorporated via the use of a block matching procedure to generate a sparse deformation field which drives the viscous fluid algorithm, This algorithm is also compared to two other popular local registration techniques, namely Gaussian convolution and the thin-plate spline warp, and is shown to produce comparable results. An improved block matching procedure is also proposed whereby a Reversible Jump Markov Chain Monte Carlo (RJMCMC) sampler is used to optimally locate grid points of interest. These grid points have a larger concentration in regions of high information and a lower concentration in regions of small information. Previous methods utilise only a uniform distribution of grid points throughout the image.

La sécurité des personnes et des équipements est un élément capital dans le domaine des transports routiers et ferroviaires. Depuis quelques années, les Passages à Niveau (PN) ont fait l’objet de davantage d'attention afin d'accroître la sécurité des usagers sur cette portion route/rail considérée comme dangereuse. Nous proposons dans cette thèse un système de vision stéréoscopique pour la détection automatique des situations dangereuses. Un tel système permet la détection et la localisation d'obstacles sur ou autour du PN. Le système de vision proposé est composé de deux caméras supervisant la zone de croisement. Nous avons développé des algorithmes permettant à la fois la détection d'objets, tels que des piétons ou des véhicules, et la localisation 3D de ces derniers. L'algorithme de détection d'obstacles se base sur l'Analyse en Composantes Indépendantes et la propagation de croyance spatio-temporelle. L'algorithme de localisation tridimensionnelle exploite les avantages des méthodes locales et globales, et est composé de trois étapes : la première consiste à estimer une carte de disparité à partir d'une fonction de vraisemblance basée sur les méthodes locales. La deuxième étape permet d'identifier les pixels bien mis en correspondance ayant des mesures de confiances élevées. Ce sous-ensemble de pixels est le point de départ de la troisième étape qui consiste à ré-estimer les disparités du reste des pixels par propagation de croyance sélective. Le mouvement est introduit comme une contrainte dans l'algorithme de localisation 3D permettant l'amélioration de la précision de localisation et l'accélération du temps de traitement. / Within the past years, railways undertakings became interested in the assessment of Level Crossings (LC) safety. We propose in this thesis an Automatic Video-Surveillance system (AVS) at LC for an automatic detection of specific events. The system allows automatically detecting and 3D localizing the presence of one or more obstacles which are motionless at the level crossing. Our research aims at developing an AVS using the passive stereo vision principles. The proposed imaging system uses two cameras to detect and localize any kind of object lying on a railway level crossing. The cameras are placed so that the dangerous zones are well (fully) monitored. The system supervises and estimates automatically the critical situations by detecting objects in the hazardous zone defined as the crossing zone of a railway line by a road or path. The AVS system is used to monitor dynamic scenes where interactions take place among objects of interest (people or vehicles). After a classical image grabbing and digitizing step, the processing is composed of the two following modules: moving and stationary objects detection and 3-D localization. The developed stereo matching algorithm stems from an inference principle based on belief propagation and energy minimization. It takes into account the advantages of local methods for reducing the complexity of the inference step achieved by the belief propagation technique which leads to an improvement in the quality of results. The motion detection module is considered as a constraint which allows improving and speeding up the 3D localization algorithm.