In this dissertation, a novel scheme performing 3D reconstruction of a scene from a 2D video sequence is presented. To this aim, first, the trajectories of the salient features in the scene are determined as a sequence of displacements via Kanade-Lukas-Tomasi tracker and Kalman filter. Then, a tentative camera trajectory with respect to a metric reference reconstruction is estimated. All frame pairs are ordered with respect to their amenability to 3D reconstruction by a metric that utilizes the baseline distances and the number of tracked correspondences between the frames. The ordered frame pairs are processed via a sequential structure-from-motion algorithm to estimate the sparse structure and camera matrices. The metric and the associated reconstruction algorithm are shown to outperform their counterparts in the literature via experiments. Finally, a mesh-based, rate-distortion efficient representation is constructed through a novel procedure driven by the error between a target image, and its prediction from a reference image and the current mesh. At each iteration, the triangular patch, whose projection on the predicted image has the largest error, is identified. Within this projected region and its correspondence on the reference frame, feature matches are extracted. The pair with the least conformance to the planar model is used to determine the vertex to be added to the mesh. The procedure is shown to outperform the dense depth-map representation in all tested cases, and the block motion vector representation, in scenes with large depth range, in rate-distortion sense.

Vast collections of historical photographs are being digitally archived and placed online, providing an objective record of the last two centuries that remains largely untapped. We propose that time-varying 3D models can pull together and index large collections of images while also serving as a tool of historical discovery, revealing new information about the locations, dates, and contents of historical images. In particular, our goal is to use computer vision techniques to tie together a large set of historical photographs of a given city into a consistent 4D model of the city: a 3D model with time as an additional dimension. To extract 4D city models from historical images, we must perform inference about the position of cameras and scene structure in both space and time. Traditional structure from motion techniques can be used to deal with the spatial problem, while here we focus on the problem of inferring temporal information: a date for each image and a time interval for which each structural element in the scene persists. We first formulate this task as a constraint satisfaction problem based on the visibility of structural elements in each image, resulting in a temporal ordering of images. Next, we present methods to incorporate real date information into the temporal inference solution. Finally, we present a general probabilistic framework for estimating all temporal variables in structure from motion problems, including an unknown date for each camera and an unknown time interval for each structural element. Given a collection of images with mostly unknown or uncertain dates, we can use this framework to automatically recover the dates of all images by reasoning probabilistically about the visibility and existence of objects in the scene. We present results for image collections consisting of hundreds of historical images of cities taken over decades of time, including Manhattan and downtown Atlanta.

This thesis describes the theoretical bases, development and testing of an integrated moving object detection framework in 2D and 3D scenes. The detection problem is analyzed in stationary and non-stationary camera sequences and different algorithms are developed for each case. Two methods are proposed in stationary camera sequences: background extraction followed by differencing and thresholding, and motion detection using optical flow field calculated by &ldquo / Kanade-Lucas Feature Tracker&rdquo / . For non-stationary camera sequences, different algorithms are developed based on the scene structure and camera motion characteristics. In planar scenes where the scene is flat or distant from the camera and/or when camera makes rotations only, a method is proposed that uses 2D parametric registration based on affine parameters of the dominant plane for independently moving object detection. A modified version of the 2D parametric registration approach is used when the scene is not planar but consists of a few number of planes at different depths, and camera makes translational motion. Optical flow field segmentation and sequential registration are the key points for this case. For 3D scenes, where the depth variation within the scene is high, a parallax rigidity based approach is developed for moving object detection. All these algorithms are integrated to form a unified independently moving object detector that works in stationary and non-stationary camera sequences and with different scene and camera motion structures. Optical flow field estimation and segmentation is used for this purpose.

Σκοπός της παρούσας εργασίας είναι η παρουσίαση των σταδίων της πιο γενικής από τις παθητικές μεθόδους τρισδιάστατης ανακατασκευής στατικών σκηνών, της δομής από κίνηση. Εκτός της χρήσης στερεοσκοπικών ζευγών εικόνων για την δημιουργία πυκνών χαρτών βάθους, παρουσιάζεται και η περίπτωση χρήσης παράλληλων εικόνων, που αποτελούν την καλύτερη λύση σε περιπτώσεις που η κύρια συνιστώσα της κίνησης της κάμερας είναι παράλληλη στον οπτικό της άξονα. Επίσης, παρουσιάζεται μια πρωτότυπη γεωμετρική μέθοδος διόρθωσης των εικόνων τόσο για την δημιουργία στερεοσκοπικών ζευγών, όσο και για την περίπτωση των παράλληλων εικόνων. Σε κάθε στάδιο της όλης διαδικασίας, περιγράφονται οι μέθοδοι εκείνες που θεωρούνται οι αντιπροσωπευτικότερες της κατηγορίας τους. / The purpose of this paper is to present the phases of the most general passive method of three dimensional reconstruction of static scenes,called structure from motion.Besides using stereo images as the final result of image rectification; we examine the case of using parallel images that represent the best solution in cases where the main component of the camera motion is parallel to the optical axis. Also we show an original geometric correction method of images both for generating stereo pairs and parallel images pairs. At each stage of the process we describe some of the known methods that can be considered representative of their class.

Thesis (MScEng (Mathematical Sciences. Applied Mathematics))--Stellenbosch University, 2008. / The medical radiation treatment facility at iThemba Labs requires a precise and robust patient positioning system. The current system makes use of an accurately calibrated multi-camera stereophotogrammetry (SPG) setup that is vulnerable to physical disruptions that invalidate system calibration. The task in this thesis is to design a vision system that can be used to verify the correct operation of the SPG system. We propose an unscented Kalman filter (UKF) based structure from motion (SFM) system for this purpose. Our SFM system does not rely on calibration information used by the SPG system and provides accurate reconstruction for verification purposes. The system is critically evaluated against a set of synthetic and real motion sequences.

I compared forest structural parameters of treated and untreated plots on a forest restoration site on Galiano Island, British Columbia. The site was replanted with Douglas-fir (Pseudotsuga menziesii (mirb.) Franco) after being intensively logged in the 1970s and then thinned in the early 2000s. I used existing baseline data from 8 permanent plots (5 treated, 3 control) and compared it with forest assessment data collected in the field in the summer of 2017. Additionally, I used 16 temporary plots (8 treated, 8 control). I assessed vegetation percentage cover by plot, coarse woody debris by plot, tree diameter, species and status (n = 846), height (n = 48) and diameter growth (n = 271). I found that treated plots showed improved measures of structural diversity like diameter growth, crown ratios and plant diversity, but I was unable to relate the increased diameter growth to the restoration treatments. My findings suggest that to create a lasting impact, restoration thinning will have to be more frequent or create larger gaps. I then reviewed the current studies with unmanned aerial vehicles (UAV) in ecological restoration. I evaluated potential use of hobbyist UAVs for small organizations and not-for-profits and found that if applied correctly, UAVs can increase the amount of available data before, during and after restoration. Reproducible and reliable results require trained personnel and calibrated sensors. UAVs can increase access to remote areas and decrease disturbance of sensitive ecosystems. Regulations, limited flight time and processing time remain important restrictions on UAV use and hobbyist UAVs have a limit availability of sensors and flight performance. Finally, I used images taken from a hobbyist UAV to assess forest structure of the restoration site on Galiano Island and compared my results with the ground measurements. I found a canopy height model (CHM) from UAV images underestimated mean tree height values for the study site on average by 10.2 metres, while also severely underestimating mean stem densities. Using a 2 metre threshold, I delineated canopy gaps which accounted for 6 % of the canopy. UAV images and the resulting CHM represent a new visualization of the study site’s structure and can be a helpful tool in the communication of restoration outcomes to a wider audience. They are not, however, sufficient for monitoring or scientific applications. / Graduate

This master thesis investigates the difficulties of constructing a depth map using one low resolution grayscale camera mounted in the front of a car. The goal is to produce a depth map in real-time to assist other algorithms in the safety system of a car. This has been shown to be difficult using the evaluated combination of camera position and choice of algorithms. The main problem is to estimate an accurate optical flow. Another problem is to handle moving objects. The conclusion is that the implementations, mainly triangulation of corresponding points tracked using a Lucas Kanade tracker, provide information of too poor quality to be useful for the safety system of a car. / I detta examensarbete undersöks svårigheterna kring att skapa en djupbild från att endast använda en lågupplöst gråskalekamera monterad framtill i en bil. Målet är att producera en djupbild i realtid som kan nyttjas i andra delar av bilens säkerhetssystem. Detta har visat sig vara svårt att lösa med den undersökta kombinationen av kameraplacering och val av algoritmer. Det huvudsakliga problemet är att räkna ut ett noggrant optiskt flöde. Andra problem härrör från objekt som rör på sig. Slutsatsen är att implementationerna, mestadels triangulering av korresponderande punktpar som följts med hjälp av en Lucas Kanade-följare, ger resultat av för dålig kvalitet för att vara till nytta för bilens säkerhetssystem.

Ces travaux de thèse constituent la première étape d'une réflexion sur la conception d'un système participatif visant la reconstruction et la visualisation de l'espace urbain de la ville de Reims au cours du temps. Dans ces travaux, nous abordons les problèmes de reconnaissance, de reconstruction et géoréférencement spatio-temporel. Ce projet s'appuie sur l'exploitation des sources historiques iconographiques et contextuelles hétérogènes et éparses, dont une collection de cartes postales anciennes et le cadastre actuel géoréférencé.Dans l'objectif d'un travail participatif, il s'avère nécessaire de procurer une aide efficace à l'utilisateur lorsqu'il apporte de nouvelles connaissances dans le système. Une solution robuste doit être apportée en raison de multiples évolutions ou déformations du modèle urbain à travers le temps. Nous avons développé une solution répondant à ce besoin. Elle s'appuie sur des briques de traitement complémentaires, en interaction avec l'utilisateur et s'insère naturellement dans une approche incrémentale de reconstruction. Nous proposons une solution permettant d'extraire, de reconstruire en 3D et de visualiser des bâtiments multi-façades présents dans les cartes postales sans disposer des dimensions réelles des bâtiments. La construction du modèle repose sur l'identification de façades 2D. Elle est obtenue au travers de l'analyse du contenu de l'image. Cette phase d'identification permet de procéder à la reconstruction de modèles 3D, d'extraire des textures 2D associées à ces modèles ainsi que d'enrichir dynamiquement notre système. Les caractéristiques retrouvées dans les images infèrent une estimation sur leur datation, et l'alignement des modèles reconstruits avec le cadastre sur le géoréférencement des bâtiments. Le système ainsi construit constitue une amorce pour la conception d'un système d'information géographique participatif 3D+T permettant aux citoyens de Reims de s'approprier l'histoire de leur ville. / This thesis is the first step toward the design of a Volunteered system for the reconstruction and visualization of urban space in the city of Reims through time. In this work, we address the problems of spatio-temporal recognition, reconstruction and georeferencing. This project relies on the use of heterogeneous and sparse iconographic and contextual historical data, particularly a collection of old postcards and the current cadastral map.With the aim of a Volunteered work, it is necessary to provide useful help to the user when bringing new knowledge into the system. A robust solution is required due to multiple changes of the urban model through time. We have developed a solution to meet those needs. This process fits in an incremental approach of reconstruction and will be completed by a user. We propose to extract, reconstruct and visualize 3D multi-façade buildings from old postcards with no knowledge on their real dimensions. The construction of the models is based on 2D façades identification. It can be obtained through image analysis. This identification allows the reconstruction of 3D models, the extraction of their associated 2D façades textures and the enhancement of the system. The features found in the images infer an estimate of their dating, and the alignment of the models with the cadastral map allows there georeferencing. The system thus constructed is a primer for the design of a Volunteered 3D+T GIS for Reims citizens to capture the history of their city.

Le problème de la stéréovision à partir de caméras multiples calibrées capturant une scène fixe est étudié depuis plusieurs décennies. Les résultats présentés dans le benchmark de stéréovision proposé par Strecha et al., attestent de la qualité des reconstructions obtenues. En particulier, les travaux du laboratoire IMAGINE, mènent à des résultats visuellement impressionnant. Aussi, il devient intéressant de calibrer des scènes de plus en plus vastes, afin d'appliquer ces algorithmes de stéréovision de façon optimale. Trois objectifs essentiels apparaissent : – La précision de la calibration doit être améliorée. En effet comme pointé par Yasutaka Furukawa, même les benchmarks de stéréovision fournissent parfois des caméras bruitées à la précision imparfaite. Un des moyen d'améliorer les résultats de stéréovision est d'augmenter la précision de la calibration. – Il est important de pouvoir prendre en compte les cycles dans le graphe des caméras de façon globale. En effet la plupart des méthodes actuelles sont séquentielles, et dérivent. Ainsi ces méthodes ne garantissent pas, pour une très grande boucle, de retrouver cette configuration cyclique, mais peuvent plutôt retrouver une configuration des caméras en spirale. Comme on calibre des réseaux d'images, de plus en plus grand, ce point est donc crucial. – Pour calibrer des réseaux d'images très grands, il convient d'avoir des algorithmes rapides. Les méthodes de calibration que nous proposons dans la première partie, permettent de calibrer des réseaux avec une précision très proche de l'état de l'art. D'autre part elle permettent de gérer les contraintes de cyclicité par le biais d'optimisations linéaires sous contraintes linéaires. Ainsi ces méthodes permettent de prendre en compte les cycles et bénéficient de la rapidité de la programmation linéaire. Enfin, la recherche en stéréovision étant arrivée à maturité, il convient de s'intéresser à l'étape suivante, à savoir la reconstruction spatio-temporelle. La méthode du laboratoire IMAGINE représentant l'état de l'art en stéréovision, il est intéressant de développer cette méthode et de l'étendre à la reconstruction spatio-temporelle, c'est-à-dire la reconstruction d'une scène dynamique capturée par un dôme de caméras. Nous verrons cette méthode dans la seconde partie de ce manuscrit / The issue of retrieving a 3D shape from a static scene captured with multiple view point calibrated cameras has been deeply studied these last decades. Results presented in the stereovision benchmark made by Strecha et al., show the high quality of state of the art methods. Particularly, works from IMAGINE laboratory lead to impressive results. So, it becomes convenient to calibrate wider and wider scenes, in order to apply these stereovision algorithms to large scale scenes. Three main objectives appear : – The calibration accuracy should be improved. As stated by Yasutaka Furukawa, even stereovision benchmarks use noisy cameras. So one obvious way to improve stereovision, is to improve camera calibration. – It is crucial to take cycles into account in cameras graph in a global way. Most of nowadays methods are sequential and so present a drift. So these methods do not offer the guarantee to retrieve the loopy configuration for a loop made of a high number of images, but retrieve a spiral configuration. As we aim to calibrate wider and wider cameras networks, this point becomes quite crucial. – To calibrate wide cameras networks, having quick and linear algorithms can be necessary. Calibration methods we propose in the first part, allow to calibrate with an accuracy close to state of the art. Moreover, we take cyclicity constraints into account in a global way, with linear optimisations under linear constraints. So these methods allow to take cycle into account and benefit from quickness of linear programming. Finally, sterovision being a well studied topic, it is convenient to concentrate on the next step, that is, spatio-temporal reconstruction. The IMAGINE' stereovision method being the state of the art, it is interesting to extend this method to spatio-temporal reconstruction, that is, dynamique scene reconstruction captured from a dome of cameras

This master's thesis focuses on 3D reconstruction of vehicles passing in front of a traffic surveillance camera. Calibration process of surveillance camera is first introduced and the relation of automatic calibration with 3D information about observed traffic is described. Furthermore, Structure from Motion, SLAM, and optical flow algorithms are presented. A set of experiments with feature matching and the Structure from Motion algorithm is carried out to examine results on images of passing vehicles. Afterwards, the Structure from Motion pipeline is modified. Instead of using SIFT features, DeepMatching algorithm is utilized to obtain quasi-dense point correspondences for the subsequent reconstruction phase. Afterwards, reconstructed models are refined by applying additional constraints specific to the vehicle reconstruction task. The resultant models are then evaluated. Lastly, observations and acquired information about the process of vehicle reconstruction are utilized to form proposals for prospective design of an entirely custom pipeline that would be specialized for 3D reconstruction of passing vehicles.