This thesis presents a 3D stereoscopic interactive fish tank system that combines the 3D stereoscopy and ¡§controller-free¡¨ components. Based on the characteristics of human vision, when seeing the objects, the left eye image and right eye image will be slightly different, one can use the intensity information and the epipolar geometry to proceed matching, and then to generate the 3D depth information. This system allows a user to use gesture to do interaction. It estimates 3D objects depth information, and uses eyes distance, distance between the user and the sensor, disparity map to calculate the virtual objects¡¦ three-dimensional coordinate, and then transforms hand and virtual objects¡¦ coordinate into the same coordinate to allow accurate interaction. The system allows users to experience the innovative multi-media interactive entertainment.

In this thesis, a novel and fast method to compute the dense disparity map of a stereo pair of images is presented. Most of the current stereo matching algorithms are ill suited for real-time matching owing to their time complexity. Methods that concentrate on providing a real-time performance, sacrifice much in accuracy. The presented method, Fast Estimation of Dense Disparity Map Using Pivot Points (FEDDUP), uses a hierarchical approach towards reduction of search space to find the correspondences. The hierarchy starts with a set of points and then it moves on to a mesh with which the edge pixels are matched. This results in a semi-global disparity map. The semi global disparity map is then used as a soft constraint to find the correspondences of the remaining points. This process delivers good real-time performance with promising accuracy.

Finding disparity maps between stereo images is a well studied topic within computer vision. While both classical and machine learning approaches exist in the literature, they frequently struggle to correctly solve the disparity in regions with low texture, sharp edges or occlusions. Finding approximate solutions to these problem areas is frequently referred to as disparity refinement, and is usually carried out separately after an initial disparity map has been generated. In the recent literature, the use of Normalized Convolution in Convolutional Neural Networks have shown remarkable results when applied to the task of stereo depth completion. This thesis investigates how well this approach performs in the case of disparity refinement. Specifically, we investigate how well such a method can improve the initial disparity maps generated by the stereo matching algorithm developed at Saab Dynamics using a rectified stereo rig. To this end, a dataset of ground truth disparity maps was created using equipment at Saab, namely a setup for structured light and the stereo rig cameras. Because the end goal is a dataset fit for training networks, we investigate an approach that allows for efficient creation of significant quantities of dense ground truth disparities. The method for generating ground truth disparities generates several disparity maps for every scene measured by using several stereo pairs. A densified disparity map is generated by merging the disparity maps from the neighbouring stereo pairs. This resulted in a dataset of 26 scenes and 104 dense and accurate disparity maps. Our evaluation results show that the chosen Normalized Convolution Network based method can be adapted for disparity map refinement, but is dependent on the quality of the input disparity map.

Fusion of multiple stereo algorithms is performed in order to obtain accurate disparity segmentation. Reliable disparity map of real-time stereo images is estimated and disparity segmentation is performed for object detection purpose. First, stereo algorithms which have high performance in real-time applications are chosen among the algorithms in the literature and three of them are implemented. Then, the results of these algorithms are fused to gain better performance in disparity estimation. In fusion process, if a pixel has the same disparity value in all algorithms, that disparity value is assigned to the pixel. Other pixels are labelled as unknown disparity. Then, unknown disparity values are estimated by a refinement procedure where neighbourhood disparity information is used. Finally, the resultant disparity map is segmented by using mean shift segmentation. The proposed method is tested in three different stereo data sets and several real stereo pairs. The experimental results indicate an improvement for the stereo analysis performance by the usage of fusion process and refinement procedure. Furthermore, disparity segmentation is realized successfully by using mean shift segmentation for detecting objects at different depth levels.

Šiame darbe yra nagrinėjama kliūčių atpažinimo vaizde problema. Ši problema susideda iš skirtumų vaizdo formavimo, vaizdo paruošimo dirbtiniam neuroniniam tinklui ir objektų klasifikavimo, naudojant dirbtinį neuroninį tinklą, uždavinių. Darbe siekiama išnagrinėti esamus vaizdo formavimo, apdorojimo ir dirbtinio neuroninio tinklo klasifikavimo būdus ir pateikti uždavinių sprendimo variantą kaip tai galima padaryti geriau. Remiantis autoriaus siūlomais sprendimais yra sukurta programinė įrangą, kuri sudaryta iš trijų modulių: skirtumų žemėlapio vaizdo formavimo, sukurtojo skirtumų žemėlapio vaizdo pirminio apdorojimo ir DNT kliūčių identifikavimo apdorotame skirtumų žemėlapio vaizde. / The problem of obstacle recognition on way is analyzed by author in this work. This problem consists of view formation, view preparation for Artificial Neural Network and object classification using neural networks tasks. It is striving to analyze the formation of view, processing of view and ways of ANN classification, and suggest the better way of task solutions in this thesis. It is compiled software using authors suggested solutions which consists of three modules: disparity map formation, filtering preparation of created one and obstacle recognition using ANN.

A stereo vision system is a robust method to sense the distance information in a scene. This research explores the stereo vision system from the fundamentals of stereo vision and the computer stereo vision algorithm to the final implementation of the system on a FPGA chip. In a stereo vision system, images are captured by a pair of stereo image sensors. The distance information can be derived from the disparities between the stereo image pair, based on the theory of binocular geometry. With the increasing focus on 3D vision, stereo vision is becoming a hot topic in the areas of computer games, robot vision and medical applications. Particularly, most stereo vision systems are expected to be used in real-time applications. In this thesis, several stereo correspondence algorithms that determine the disparities between stereo image pair are examined. The algorithms can be categorized into global stereo algorithms and local stereo algorithms depending on the optimization techniques. The global algorithms examined are the Dynamic Time Warp (DTW) algorithm and the DTW with quantization algorithm, while the local algorithms examined are the window based Sum of Squared Differences (SSD), Sum of Absolute Differences (SAD) and Census transform correlation algorithms. With analysis among them, the window based SAD correlation algorithm is proposed for implementation on a FPGA platform. The proposed algorithm is implemented onto an Altera DE2 board featuring an Altera Cyclone II 2C35 FPGA. The implemented module of the algorithm is simulated using ModelSim-Altera to verify the correctness of its functionality. Along with a pair of stere image sensors and a LCD monitor, a stereo vision system is built. The entire system realizes a real-time video frame rate of 16.83 frames per second with an image resolution of 640 by 480 and produces disparity maps in which the objects are clearly distinguished by their relative distance information.

En vision industrielle, de nombreuses applications de mesure et de contrôle qualité évoluent vers des problématiques tri-dimensionnelles. Les systèmes de stéréovision sont des solutions technologiques qui attirent les industriels par leur simplicité mécanique. Deux caméras statiques disposées à des endroits stratégiques peut s'avérer suffisantes pour répondre à cette problématique bien que les contraintes industrielles imposent de respecter des temps de traitement courts et des mesures précises. La diversité des applications nous amènent à envisager deux approches afin de répondre à deux types d'application. La première technique consiste en la reconstruction 3D à partir de paires de points images qui se correspondent dans les deux images. Elle est destinée à répondre à la problématique de mesure 3D. Les méthodes de calibration monoculaire et de calcul 3D par triangulation sont la base de la reconstruction 3D. Nous étudions la précision de mesure et son évolution selon la pose du système de capture par rapport à la scène observée. La seconde technique consiste à construire des images de disparité afin de répondre à des problématiques de construction de profil et d'analyse de défaut. La contrainte d'alignement des caméras, nécessaire pour accélérer le processus de mise en correspondance, implique d'utiliser des méthodes de calibration stéréoscopique et de rectification des images. Nous étudions l'impact de l'alignement sur la qualité de la rectification. La production de carte dense de disparité se base sur les techniques de stéréo-corrélation. Nous montrons les limites de l'utilisation d'un noyau de corrélation carré et proposons une alternative par production de deux cartes denses de disparité à partir de deux noyaux mono-directionnels, améliorant la mesure de disparité sur les zones de contours et d'occultations. / In industrial vision, many applications for measuring and quality control are moving to three-dimensional problems. Stereovision systems are technological solutions that attract industry by their mechanical simplicity. Two static cameras placed at strategic locations may be sufficient to address this problem although the industrial constraints imposed to respect a short processing time and precise measurements. The diversity of applications lead us to consider two approaches to resolve the two types of application. The first technique consists in the 3D reconstruction from pairs of image points which correspond in both images. It is intended to address the problem of 3D measurement. The methods of monocular calibration and 3D triangulation are the basis of 3D reconstruction. We study the accuracy and its evolution according to the capture system pose compared to the observed scene. The second technique is to construct disparity maps to address problems of building profile and default analysis. The alignment constraint of cameras needed to accelerate the process of matching involves the use of methods of stereoscopic calibration and image rectification. We study the impact of alignment on the quality of the rectification. The production of dense disparity map is based on the stereo-correlation techniques. We show the limits of the use of a squared correlation kernel and propose an alternative production of two dense disparity maps from two mono-directional kernels, improving the measurement of disparity around edges and occlusions.

The master thesis focuses on disparity map estimation using convolutional neural network. It discusses the problem of using convolutional neural networks for image comparison and disparity computation from stereo image as well as existing approaches of solutions for given problem. It also proposes and implements system that consists of convolutional neural network that measures the similarity between two image patches, and filtering and smoothing methods to improve the result disparity map. Experiments and results show, that the most quality disparity maps are computed using CNN on input patches with the size of 9x9 pixels combined with matching cost agregation and correction algorithm and bilateral filter.

The thesis deals with head pose estimation in stereo data. The theoretical part provides the basis for understanding the geometry of the camera, its parameters and the method of calibration. The following describes the principles of stereo analysis and creating of disparity maps. In the research section, the methods used for head pose modelling are presented and an analysis of selected published articles is given. In the course of the master’s thesis, a system of two cameras for stereoscopic acquisition of motion of the head was designed and several measurements were carried out. The obtained data was prepared for creation of disparity maps and further processing. Based on the detection of facial features, in particular the inner and outer corners of the eyes and corners of the mouth, and their correspondences, a simple geometric model in shape of triangle was created to illustrate the inclination of the facial plane in space. By computing the angle of inclination in three axes, the current head pose is obtained. Motion is modelled by tracking detected points during video sequences.

Project shows calibration stereoscopic sensor. Also describes basic methods stereo-corespodation using library OpenCV. Project contains calculations of disparity maps on CPU or graphic card(using library OpenCL).