This thesis explores efficient techniques for generating 3D reconstructions from imagery. Non-linear optimization is one of the core techniques used when computing a reconstruction and is a computational bottleneck for large sets of images. Since non-linear optimization requires a good initialization to avoid getting stuck in local minima, robust systems for generating reconstructions from images build up the reconstruction incrementally. A hierarchical approach is to split up the images into small subsets, reconstruct each subset independently and then hierarchically merge the subsets. Rigidly locking together portions of the reconstructions reduces the number of parameters needed to represent them when merging, thereby lowering the computational cost of the optimization. We present two techniques that involve optimizing with parts of the reconstruction rigidly locked together. In the first, we start by rigidly grouping the cameras and scene features from each of the reconstructions being merged into separate groups. Cameras and scene features are then incrementally unlocked and optimized until the reconstruction is close to the minimum energy. This technique is most effective when the influence of the new measurements is restricted to a small set of parameters. Measurements that stitch together weakly coupled portions of the reconstruction, though, tend to cause deformations in the low error modes of the reconstruction and cannot be efficiently incorporated with the previous technique. To address this, we present a spectral technique for clustering the tightly coupled portions of a reconstruction into rigid groups. Reconstructions partitioned in this manner can closely mimic the poorly conditioned, low error modes, and therefore efficiently incorporate measurements that stitch together weakly coupled portions of the reconstruction. We explain how this technique can be used to scalably and efficiently generate reconstructions from large sets of images.

In this dissertation, a novel scheme performing 3D reconstruction of a scene from a 2D video sequence is presented. To this aim, first, the trajectories of the salient features in the scene are determined as a sequence of displacements via Kanade-Lukas-Tomasi tracker and Kalman filter. Then, a tentative camera trajectory with respect to a metric reference reconstruction is estimated. All frame pairs are ordered with respect to their amenability to 3D reconstruction by a metric that utilizes the baseline distances and the number of tracked correspondences between the frames. The ordered frame pairs are processed via a sequential structure-from-motion algorithm to estimate the sparse structure and camera matrices. The metric and the associated reconstruction algorithm are shown to outperform their counterparts in the literature via experiments. Finally, a mesh-based, rate-distortion efficient representation is constructed through a novel procedure driven by the error between a target image, and its prediction from a reference image and the current mesh. At each iteration, the triangular patch, whose projection on the predicted image has the largest error, is identified. Within this projected region and its correspondence on the reference frame, feature matches are extracted. The pair with the least conformance to the planar model is used to determine the vertex to be added to the mesh. The procedure is shown to outperform the dense depth-map representation in all tested cases, and the block motion vector representation, in scenes with large depth range, in rate-distortion sense.

Vast collections of historical photographs are being digitally archived and placed online, providing an objective record of the last two centuries that remains largely untapped. We propose that time-varying 3D models can pull together and index large collections of images while also serving as a tool of historical discovery, revealing new information about the locations, dates, and contents of historical images. In particular, our goal is to use computer vision techniques to tie together a large set of historical photographs of a given city into a consistent 4D model of the city: a 3D model with time as an additional dimension. To extract 4D city models from historical images, we must perform inference about the position of cameras and scene structure in both space and time. Traditional structure from motion techniques can be used to deal with the spatial problem, while here we focus on the problem of inferring temporal information: a date for each image and a time interval for which each structural element in the scene persists. We first formulate this task as a constraint satisfaction problem based on the visibility of structural elements in each image, resulting in a temporal ordering of images. Next, we present methods to incorporate real date information into the temporal inference solution. Finally, we present a general probabilistic framework for estimating all temporal variables in structure from motion problems, including an unknown date for each camera and an unknown time interval for each structural element. Given a collection of images with mostly unknown or uncertain dates, we can use this framework to automatically recover the dates of all images by reasoning probabilistically about the visibility and existence of objects in the scene. We present results for image collections consisting of hundreds of historical images of cities taken over decades of time, including Manhattan and downtown Atlanta.

This thesis describes the theoretical bases, development and testing of an integrated moving object detection framework in 2D and 3D scenes. The detection problem is analyzed in stationary and non-stationary camera sequences and different algorithms are developed for each case. Two methods are proposed in stationary camera sequences: background extraction followed by differencing and thresholding, and motion detection using optical flow field calculated by &ldquo / Kanade-Lucas Feature Tracker&rdquo / . For non-stationary camera sequences, different algorithms are developed based on the scene structure and camera motion characteristics. In planar scenes where the scene is flat or distant from the camera and/or when camera makes rotations only, a method is proposed that uses 2D parametric registration based on affine parameters of the dominant plane for independently moving object detection. A modified version of the 2D parametric registration approach is used when the scene is not planar but consists of a few number of planes at different depths, and camera makes translational motion. Optical flow field segmentation and sequential registration are the key points for this case. For 3D scenes, where the depth variation within the scene is high, a parallax rigidity based approach is developed for moving object detection. All these algorithms are integrated to form a unified independently moving object detector that works in stationary and non-stationary camera sequences and with different scene and camera motion structures. Optical flow field estimation and segmentation is used for this purpose.

Το αντικείμενο της εργασίας είναι η τρισδιάστατη ανακατασκευή ενός αντικειμένου ή ενός χώρου, μέσα από τουλάχιστο δύο φωτογραφίες του. Το θέμα της εργασίας είναι μέρος του τομέα της Υπολογιστικής Όρασης, που έχει μεγάλη άνθιση τα τελευταία χρόνια λόγω των πολλών εφαρμογών, όπου η γνώση της τρισδιάστατης δομής ενός αντικειμένου ή ενός χώρου κρίνεται απαραίτητη. Βέβαια, συνέβαλε σε αυτό και η ραγδαία εξέλιξη των ηλεκτρονικών υπολογιστών, με αποτέλεσμα να είναι δυνατή η ακριβής και ποιοτική απεικόνιση σύνθετων τρισδιάστατων σκηνών σε πραγματικό χρόνο, μέσω κατάλληλων αλγορίθμων. Η τρισδιάστατη ανακατασκευή ενός αντικειμένου ή ενός χώρου, από φωτογραφίες ή βίντεο αποτελεί ένα ενδιαφέρον και εντυπωσιακό θέμα με πολλές εφαρμογές και πολύ ενθαρρυντικά αποτελέσματα. Αυτά αποτέλεσαν ουσιαστικά και το έναυσμα για την ενασχόλησή μου με τον τομέα της Υπολογιστικής Όρασης και την επιλογή του θέματος της εργασίας. Οι εφαρμογές που αναπτύσσονται στα πλαίσια της εργασίας παρέχουν οπτικά ευχάριστα αποτελέσματα και έχουν μεγάλη προσαρμοστικότητα και ευελιξία στης διάφορες συνθήκες φωτογράφησης ή λήψης βίντεο. Το σημαντικό, λοιπόν, είναι ότι δεν χρειάζεται απαραίτητα περιβάλλον εργαστηρίου για την λήψη των δεδομένων, δηλαδή των εικόνων. Προκύπτουν καλά αποτελέσματα ακόμα και με εικόνες που λήφθηκαν μέσω μίας απλής φορητής φωτογραφικής κάμερας, χωρίς τρίποδα για στήριξη. Αρκεί απλά να δημιουργήσουμε τις προϋποθέσεις για μικρή κίνηση της κάμερας μεταξύ των λήψεων των εικόνων. Στην παρούσα εργασία παρουσιάζονται και εξετάζονται διεξοδικά όλα τα θέματα που αφορούν την τρισδιάστατη οπτικοποίηση των αντικειμένων. Αρχικά, στις παραγράφους 2.1 έως 3.2, αναλύεται η θεωρία των δύο κυριότερων μεθόδων της "Δομής και Κίνησης" και της "Στερεοσκοπικής Όρασης" . Στην συνέχεια στις παραγράφους 3.3 και 3.4 αναπτύσσεται η μεθοδολογία που ακολουθείται από τις εφαρμογές της Δομής και Κίνησης , ενώ της Στερεοσκοπική Όρασης αναπτύσσεται στην 3.5 . Στην μέθοδο της Δομής και Κίνησης περικλείονται δύο περιπτώσεις. Η πρώτη είναι η μη βαθμονομημένη περίπτωση και η δεύτερη είναι η βαθμονομημένη. Στην δεύτερη, λοιπόν, προηγείται η βαθμονόμηση της κάμερας, οπότε είναι γνωστές εκ των προτέρων οι παράμετροι της κάμερας . Η υλοποίηση των αλγορίθμων γίνεται στο παράρτημα, με την βοήθεια του περιβάλλοντος αριθμητικής υπολογιστικής της προγραμματιστικής γλώσσας Matlab. Τέλος, στο τέταρτο κεφάλαιο, δίνονται κάποια παραδείγματα ανακατασκευών που αποδεικνύουν την αποτελεσματικότητα των αλγορίθμων της υλοποίησης. Τόσο η θεωρία, όσο και οι αλγόριθμοι που παρουσιάζονται στην παρούσα εργασία καλύπτουν πλήρως τις απαραίτητες γνώσεις για την υλοποίηση της τρισδιάστατης αναπαράστασης. Συσσωρεύτηκαν πληροφορίες από δύο μεθόδους, δηλαδή της Δομής και Κίνησης αλλά και της Στερεοσκοπικής Όρασης, οι οποίες συνδυασμένες δίνουν βέλτιστα και αρτιότερα αποτελέσματα. Πρωταρχικός στόχος της εργασίας είναι η ανάδειξη των δυνατοτήτων που παρέχουν οι συγκεκριμένες μέθοδοι. Από την άλλη μεριά, η υλοποίηση των δύο μεθόδων και κατά συνέπεια των αλγόριθμων, αποτελούν μία αρκετά καλή βάση για περαιτέρω ανάπτυξη και προώθηση για μελλοντική έρευνα στον εν λόγω τομέα. Ούτως ή άλλως, τα τελευταία χρόνια, η ερεύνα που αφορά την Υπολογιστική Όραση έχει αποδώσει και έχει δημιουργήσει ικανοποιητικότατα αποτελέσματα. Οπότε στο μέλλον αναμένονται ισχυρότεροι αλγόριθμοι, βελτιώσεις αλλά και πολλές εφαρμογές στους εξελισσόμενους τομείς της ηλεκτρονικής και όχι μόνο. / The purpose of this thesis is the three-dimensional reconstruction of an object or a space, through at least two photos. The theme is part of the field of Computer Vision, which has known great development in the recent years due to the many applications, where the knowledge of the three-dimensional structure of an object or a space is necessary. Of course to this development contributed also the rapid evolution of computers, making possible the accurate and high quality display of complex three-dimensional scenes in real time, through appropriate algorithms. The three-dimensional reconstruction of an object or a space, using photos or video, is an interesting and impressive subject with many applications and very encouraging results. This was basically what intrigued me to involve with the field of Computer Vision and choose the topic of this thesis. The applications in this thesis provide visually pleasant results and have great adaptability and flexibility in various conditions of shooting and making of videos. What is important, therefore, is that a laboratory environment to obtain the data, meaning images, is not necessary. The results are satisfactory even with pictures taken by a simple handheld camera, without the use of a tripod. It’s enough if we just create the proper conditions for a small camera movement between the shots. The present thesis presents and discusses thoroughly all the subjects related with the three-dimensional visualization of objects. First in paragraphs 2.1 to 3.2, is analyzed the theory of the two most important methods, of "Structure and Motion" and of "Stereo Vision". Then in paragraphs 3.3 and 3.4 is discussed the methodology followed by the applications of Structure and Motion , and this of Stereo Vision is discussed in paragraph 3.5. The method of Structure and Motion encloses two cases. The first is the non-calibrated case and the second is the calibrated case. In the calibrated case the calibration of the camera comes first, so the parameters of the camera are known in advance. The implementation of the algorithms is in the Annex, with the help of the numerical computing environment of the programming language Matlab. Finally, in chapter four, are given same examples of reconstructions that demonstrate the effectiveness of the algorithms of implementation. Both the theory and the algorithms presented in this thesis cover fully the necessary knowledge for the materialization of the three-dimensional representation. The information are is accumulated by two methods, this of Structure and Motion and this of Stereo Vision, which combined give the best and most complete results. Primary objective of this thesis is to highlight the possibilities offered by these methods. On the other hand, the implementation of these two methods and thus the algorithms is a good basis for further development and promotion for future research in this field. Anyway, in recent years, the research on Computer Vision has given great results. So in the future are expected stronger algorithms, improvements but also many applications relating with the developing sector of electronics and more.

Σκοπός της παρούσας εργασίας είναι η παρουσίαση των σταδίων της πιο γενικής από τις παθητικές μεθόδους τρισδιάστατης ανακατασκευής στατικών σκηνών, της δομής από κίνηση. Εκτός της χρήσης στερεοσκοπικών ζευγών εικόνων για την δημιουργία πυκνών χαρτών βάθους, παρουσιάζεται και η περίπτωση χρήσης παράλληλων εικόνων, που αποτελούν την καλύτερη λύση σε περιπτώσεις που η κύρια συνιστώσα της κίνησης της κάμερας είναι παράλληλη στον οπτικό της άξονα. Επίσης, παρουσιάζεται μια πρωτότυπη γεωμετρική μέθοδος διόρθωσης των εικόνων τόσο για την δημιουργία στερεοσκοπικών ζευγών, όσο και για την περίπτωση των παράλληλων εικόνων. Σε κάθε στάδιο της όλης διαδικασίας, περιγράφονται οι μέθοδοι εκείνες που θεωρούνται οι αντιπροσωπευτικότερες της κατηγορίας τους. / The purpose of this paper is to present the phases of the most general passive method of three dimensional reconstruction of static scenes,called structure from motion.Besides using stereo images as the final result of image rectification; we examine the case of using parallel images that represent the best solution in cases where the main component of the camera motion is parallel to the optical axis. Also we show an original geometric correction method of images both for generating stereo pairs and parallel images pairs. At each stage of the process we describe some of the known methods that can be considered representative of their class.

Thesis (MScEng (Mathematical Sciences. Applied Mathematics))--Stellenbosch University, 2008. / The medical radiation treatment facility at iThemba Labs requires a precise and robust patient positioning system. The current system makes use of an accurately calibrated multi-camera stereophotogrammetry (SPG) setup that is vulnerable to physical disruptions that invalidate system calibration. The task in this thesis is to design a vision system that can be used to verify the correct operation of the SPG system. We propose an unscented Kalman filter (UKF) based structure from motion (SFM) system for this purpose. Our SFM system does not rely on calibration information used by the SPG system and provides accurate reconstruction for verification purposes. The system is critically evaluated against a set of synthetic and real motion sequences.

I compared forest structural parameters of treated and untreated plots on a forest restoration site on Galiano Island, British Columbia. The site was replanted with Douglas-fir (Pseudotsuga menziesii (mirb.) Franco) after being intensively logged in the 1970s and then thinned in the early 2000s. I used existing baseline data from 8 permanent plots (5 treated, 3 control) and compared it with forest assessment data collected in the field in the summer of 2017. Additionally, I used 16 temporary plots (8 treated, 8 control). I assessed vegetation percentage cover by plot, coarse woody debris by plot, tree diameter, species and status (n = 846), height (n = 48) and diameter growth (n = 271). I found that treated plots showed improved measures of structural diversity like diameter growth, crown ratios and plant diversity, but I was unable to relate the increased diameter growth to the restoration treatments. My findings suggest that to create a lasting impact, restoration thinning will have to be more frequent or create larger gaps. I then reviewed the current studies with unmanned aerial vehicles (UAV) in ecological restoration. I evaluated potential use of hobbyist UAVs for small organizations and not-for-profits and found that if applied correctly, UAVs can increase the amount of available data before, during and after restoration. Reproducible and reliable results require trained personnel and calibrated sensors. UAVs can increase access to remote areas and decrease disturbance of sensitive ecosystems. Regulations, limited flight time and processing time remain important restrictions on UAV use and hobbyist UAVs have a limit availability of sensors and flight performance. Finally, I used images taken from a hobbyist UAV to assess forest structure of the restoration site on Galiano Island and compared my results with the ground measurements. I found a canopy height model (CHM) from UAV images underestimated mean tree height values for the study site on average by 10.2 metres, while also severely underestimating mean stem densities. Using a 2 metre threshold, I delineated canopy gaps which accounted for 6 % of the canopy. UAV images and the resulting CHM represent a new visualization of the study site’s structure and can be a helpful tool in the communication of restoration outcomes to a wider audience. They are not, however, sufficient for monitoring or scientific applications. / Graduate

This master thesis investigates the difficulties of constructing a depth map using one low resolution grayscale camera mounted in the front of a car. The goal is to produce a depth map in real-time to assist other algorithms in the safety system of a car. This has been shown to be difficult using the evaluated combination of camera position and choice of algorithms. The main problem is to estimate an accurate optical flow. Another problem is to handle moving objects. The conclusion is that the implementations, mainly triangulation of corresponding points tracked using a Lucas Kanade tracker, provide information of too poor quality to be useful for the safety system of a car. / I detta examensarbete undersöks svårigheterna kring att skapa en djupbild från att endast använda en lågupplöst gråskalekamera monterad framtill i en bil. Målet är att producera en djupbild i realtid som kan nyttjas i andra delar av bilens säkerhetssystem. Detta har visat sig vara svårt att lösa med den undersökta kombinationen av kameraplacering och val av algoritmer. Det huvudsakliga problemet är att räkna ut ett noggrant optiskt flöde. Andra problem härrör från objekt som rör på sig. Slutsatsen är att implementationerna, mestadels triangulering av korresponderande punktpar som följts med hjälp av en Lucas Kanade-följare, ger resultat av för dålig kvalitet för att vara till nytta för bilens säkerhetssystem.

L'objectif principal de cette thèse est de développer des algorithmes génériques d'estimation du mouvement et de la structure à partir d'images de scènes prises par différents types de systèmes d'acquisition centrale : caméra perspective,fish-eye et systèmes catadioptriques, notamment. En supposant que la correspondance entre les pixels de l'image et les lignes de vue dans l'espace est connue, nous travaillons sur des images sphériques, plutôt que sur des images planes (projection des images sur la sphère unitaire), ce qui nous permet de considérer des points sur une vue mieux adaptée aux images omnidirectionnelles et d'utiliser un modèle générique valable pour tous les capteurs centraux. Dans la première partie de cette thèse, nous développons une approche générique de mise en correspondance simple de lignes à partir d'images de scènes urbaines ou péri-urbaines sous la contrainte d'un faible déplacement du capteur,ainsi qu'une contrainte rapide et originale pour apparier des lignes d'un environnement plan par morceaux, indépendante du mouvement de la caméra centrale. Ensuite, nous introduisons une méthode unique et effcace pour estimer le recouvrement entre deux segments sur des images perspectives, diminuant considérablement le temps global de calcul par rapport aux algorithmes connus.Enfin, dans la dernière partie de cette thèse, nous développons un algorithme d'estimation du mouvement et de reconstruction de surfaces pour les scènes planes par morceaux applicable à toutes sortes d'images centrales, à partir de deux vues uniquement et ne nécessitant qu'un nombre minime de correspondances de ligne. Pour démontrer les performances de ces algorithmes, nous les avons expérimentés avec diverses images réelles acquises à partir d'une caméra perspective,une lentille fish-eye, et deux différents types de capteurs paracatadioptriques(l'un est composé d'un miroir simple, et l'autre d'un miroir double). / The primary goal of this thesis is to develop generic motion and structure algorithms for images taken from constructed scenes by various types of central imaging systems including perspective, fish-eye and catadioptric systems. As-suming that the mapping between the image pixels and their 3D rays in space is known, instead of image planes, we work on image spheres (projection of the images on a unit sphere) which enable us to present points over the entire viewsphere suitable for presenting omnidirectional images. In the first part of this thesis, we develop a generic and simple line matching approach for images taken from constructed scenes under a short baseline motion as well as a fast and original geometric constraint for matching lines in planar constructed scenes insensible to the motion of the camera for all types of centralimages including omnidirectional images.Next, we introduce a unique and efficient way of computing overlap between two segments on perspective images which considerably decreases the over all computational time of a segment-based motion estimation and reconstruction algorithm. Finally in last part of this thesis, we develop a simple motion estima-tion and surface reconstruction algorithm for piecewise planar scenes applicable to all kinds of central images which uses only two images and is based on mini-mum line correspondences.To demonstrate the performance of these algorithms we experiment withvarious real images taken by a simple perspective camera, a fish-eye lens, and two different kinds of paracatadioptric sensors, the first one is a folded catadioptric camera and the second one is a classic paracatadioptric system composed of a parabolic mirror in front of a telecentric lens.