La perception est un point clé pour le fonctionnement d'un véhicule autonome ou même pour un véhicule fournissant des fonctions d'assistance. Un véhicule observe le monde externe à l'aide de capteurs et construit un modèle interne de l'environnement extérieur. Il met à jour en continu ce modèle de l'environnement en utilisant les dernières données des capteurs. Dans ce cadre, la perception peut être divisée en deux étapes : la première partie, appelée SLAM (Simultaneous Localization And Mapping) s'intéresse à la construction d'une carte de l'environnement extérieur et à la localisation du véhicule hôte dans cette carte, et deuxième partie traite de la détection et du suivi des objets mobiles dans l'environnement (DATMO pour Detection And Tracking of Moving Objects). En utilisant des capteurs laser de grande précision, des résultats importants ont été obtenus par les chercheurs. Cependant, avec des capteurs laser de faible résolution et des données bruitées, le problème est toujours ouvert, en particulier le problème du DATMO. Dans cette thèse nous proposons d'utiliser la vision (mono ou stéréo) couplée à un capteur laser pour résoudre ce problème. La première contribution de cette thèse porte sur l'identification et le développement de trois niveaux de fusion. En fonction du niveau de traitement de l'information capteur avant le processus de fusion, nous les appelons "fusion bas niveau", "fusion au niveau de la détection" et "fusion au niveau du suivi". Pour la fusion bas niveau, nous avons utilisé les grilles d'occupations. Pour la fusion au niveau de la détection, les objets détectés par chaque capteur sont fusionnés pour avoir une liste d'objets fusionnés. La fusion au niveau du suivi requiert le suivi des objets pour chaque capteur et ensuite on réalise la fusion entre les listes d'objets suivis. La deuxième contribution de cette thèse est le développement d'une technique rapide pour trouver les bords de route à partir des données du laser et en utilisant cette information nous supprimons de nombreuses fausses alarmes. Nous avons en effet observé que beaucoup de fausses alarmes apparaissent sur le bord de la route. La troisième contribution de cette thèse est le développement d'une solution complète pour la perception avec un capteur laser et des caméras stéréo-vision et son intégration sur un démonstrateur du projet européen Intersafe-2. Ce projet s'intéresse à la sécurité aux intersections et vise à y réduire les blessures et les accidents mortels. Dans ce projet, nous avons travaillé en collaboration avec Volkswagen, l'Université Technique de Cluj-Napoca, en Roumanie et l'INRIA Paris pour fournir une solution complète de perception et d'évaluation des risques pour le démonstrateur de Volkswagen.

An isolated wing-tip vortex from a square-tipped NACA 0012 wing at an angle of attack of 5 degrees was studied in a water tunnel at a chord based Reynolds number of approximately 24000. Measurements were taken using stereo particle image velocimetry at three measurement planes downstream of the wing under each of three freestream turbulence conditions. The amplitude of wandering of the vortex axis increased with increasing distance downstream of the wing and with increasing freestream turbulence intensity. The magnitude of the peak azimuthal velocity decreased with increasing distance from the wing as well as with increases in the freestream turbulence intensity. The streamwise velocity in the vortex core was less than the freestream velocity in all cases. Time resolved histories of the instantaneous waveform shape and location of the vortex axis were determined from sequences of images of fluorescent dye released from the wing.

In mobile robot applications, it is an important issue for a robot to know where it is. Accurate localization becomes crucial for navigation and map building applications because both route to follow and positions of the objects to be inserted into the map highly depend on the position of the robot in the environment. For localization, the robot uses the measurements that it takes by various devices such as laser rangefinders, sonars, odometry devices and vision. Generally these devices give the distances of the objects in the environment to the robot and proceesing these distance information, the robot finds its location in the environment. In this thesis, two vision-based robot localization algorithms are implemented. The first algorithm uses artificial landmarks as the objects around the robot and by measuring the positions of these landmarks with respect to the camera system, the robot locates itself in the environment. Locations of these landmarks are known. The second algorithm instead of using artificial landmarks, estimates its location by measuring the positions of the objects that naturally exist in the environment. These objects are treated as natural landmarks and locations of these landmarks are not known initially. A three-wheeled robot base on which a stereo camera system is mounted is used as the mobile robot unit. Processing and control tasks of the system is performed by a stationary PC. Experiments are performed on this robot system. The stereo camera system is the measurement device for this robot.

La visió és probablement el nostre sentit més dominant a partir del qual derivem la majoria d'informació del món que ens envolta. A través de la visió podem percebre com són les coses, on són i com es mouen. En les imatges que percebem amb el nostre sistema de visió podem extreure'n característiques com el color, la textura i la forma, i gràcies a aquesta informació som capaços de reconèixer objectes fins i tot quan s'observen sota unes condicions totalment diferents. Per exemple, som capaços de distingir un mateix objecte si l'observem des de diferents punts de vista, distància, condicions d'il·luminació, etc.La Visió per Computador intenta emular el sistema de visió humà mitjançant un sistema de captura d'imatges, un ordinador, i un conjunt de programes. L'objectiu desitjat no és altre que desenvolupar un sistema que pugui entendre una imatge d'una manera similar com ho realitzaria una persona. Aquesta tesi es centra en l'anàlisi de la textura per tal de realitzar el reconeixement de superfícies. La motivació principal és resoldre el problema de la classificació de superfícies texturades quan han estat capturades sota diferents condicions, com ara distància de la càmera o direcció de la il·luminació. D'aquesta forma s'aconsegueix reduir els errors de classificació provocats per aquests canvis en les condicions de captura.En aquest treball es presenta detalladament un sistema de reconeixement de textures que ens permet classificar imatges de diferents superfícies capturades en diferents condicions. El sistema proposat es basa en un model 3D de la superfície (que inclou informació de color i forma) obtingut mitjançant la tècnica coneguda com a 4-Source Colour Photometric Stereo (CPS). Aquesta informació és utilitzada posteriorment per un mètode de predicció de textures amb l'objectiu de generar noves imatges 2D de les textures sota unes noves condicions. Aquestes imatges virtuals que es generen seran la base del nostre sistema de reconeixement, ja que seran utilitzades com a models de referència per al nostre classificador de textures.El sistema de reconeixement proposat combina les Matrius de Co-ocurrència per a l'extracció de característiques de textura, amb la utilització del Classificador del veí més proper. Aquest classificador ens permet al mateix temps aproximar la direcció d'il·luminació present en les imatges que s'utilitzen per testejar el sistema de reconeixement. És a dir, serem capaços de predir l'angle d'il·luminació sota el qual han estat capturades les imatges de test. Els resultats obtinguts en els diferents experiments que s'han realitzat demostren la viabilitat del sistema de predicció de textures, així com del sistema de reconeixement. / This thesis is concerned with the application of texture analysis to discriminate between textured surfaces. The main motivation is the problem of classifying textured surfaces imaged under varying geometries, i.e. distance from the sensor and illumination direction, as well as the necessity of finding reliable methods of reducing classification errors caused by changes in the geometry's properties. In texture analysis one must distinguish between image texture and surface texture. Image texture is what appears in the 2D image of a physical object, while surface texture refers to the variation of the physical and geometric properties of the imaged surface which give rise to the image texture. Changes in the imaging geometry can significantly alter the appearance of the surface, implying significant variations in the image texture. And one still has to perform the task of recognition from the image texture. In this thesis, after analysing different strategies, we integrate the surface texture information derived by colour photometric stereo (CPS) into a complete model-based texture classification system. Photometric stereo is the technique which allows us to obtain surface texture information from a few images of the same surface imaged under various illumination directions. Basically, the main idea of our strategy consists of creating, by means of the surface texture information, a virtual' database of image textures against which we compare unknown test images in order to classify them. Note that we do not use the surface texture information directly to perform classification, but we use it to create new images which are the references for our training and classification process. Furthermore, the classification system allows us to guess the approximate direction of the illumination used to capture the test images.The proposed prediction methods, as well as the model-based texture classification system, are tested and evaluated. A set of real surface textures containing a wide variety of relatively smooth and very rough surfaces are used in this thesis as our image database.

Η παρούσα διπλωματική εργασία αναπτύχθηκε στα πλαίσια των προπτυχιακών σπουδών του τμήματος Ηλεκτρολόγων Μηχανικών και Τεχνολογίας Υπολογιστών του Πανεπιστημίου Πατρών. Θέμα της είναι η τρισδιάστατη ανακατασκευή του χώρου από τουλάχιστον δύο φωτογραφίες του και αποτελεί μέρος του τομέα της Υπολογιστικής Όρασης. Συγκεκριμένα αναλύεται διεξοδικά η περίπτωση της στερεοσκοπικής όρασης, στην οποία η κάμερα μεταξύ δύο διαδοχικών λήψεων της ίδιας σκηνής, έχει μηδενική σχετική περιστροφή ως προς την αρχική της θέση και μικρή μετατόπιση, περίπου 5 εκατοστά. Με τον τρόπο αυτό, προσπαθούμε να προσομοιώσουμε τη λειτουργία της ανθρώπινης όρασης καθώς πολλές εφαρμογές της Τεχνητής Νοημοσύνης το κρίνουν απαραίτητο. Είναι λογικό ότι ο κάθε άνθρωπος θεωρεί τη στερεοσκοπική όραση αυτονόητη γιατί κινείται στον τρισδιάστατο κόσμο. Όταν αυτός όμως καταγράφεται από μία κάμερα, αυτόματα περνάει στο δισδιάστατο επίπεδο. Και πάλι είναι δυνατόν να εξάγουμε πληροφορίες βάθους από μία μόνο εικόνα, όμως γίνεται καθαρά εμπειρικά και βασίζεται στη σύγκριση διάφορων υφών, σχημάτων και μεγεθών. Ο ηλεκτρονικός υπολογιστής αναγνωρίζει την εικόνα σαν ένα οποιοδήποτε αρχείο. Δεν μπορεί να εξάγει κανένα συμπέρασμα για το τι απεικονίζει στον πραγματικό κόσμο. Χρειάζεται το συνδυασμό τουλάχιστον δύο εικόνων της ίδιας σκηνής από διαφορετικές θέσεις για να μπορέσει να αναγνωρίσει για παράδειγμα το βάθος της σκηνής που απεικονίζεται. Αυτή τη διαδικασία περιγράφει αναλυτικά η εργασία. Στο πρώτο κεφάλαιο εισάγουμε την έννοια και τη χρησιμότητα της στερεοσκοπικής όρασης. Στο δεύτερο κεφάλαιο παρουσιάζονται οι βασικές αρχές της προβολικής γεωμετρίας. Στο τρίτο κεφάλαιο αναφερόμαστε στη μοντελοποίηση της κάμερας και τις παραμέτρους που τη χαρακτηρίζουν. Στο τέταρτο κεφάλαιο αναλύεται η διαδικασία της βαθμονόμησης της κάμερας. Στο πέμπτο κεφάλαιο εξηγείται η διαδικασία αντιστοίχησης των σημείων ενδιαφέροντος στις δύο εικόνες. Στο έκτο κεφάλαιο αναλύονται οι βασικές αρχές της επιπολικής γεωμετρίας. Στο έβδομο κεφάλαιο παρουσιάζεται η πειραματική διαδικασία για την εύρεση του βάθους της σκηνής. Στο όγδοο κεφάλαιο παρουσιάζεται συνοπτικά η τρισδιάστατη ανακατασκευή του χώρου και παρουσιάζονται τα αντίστοιχα πειραματικά αποτελέσματα. Στο ένατο κεφάλαιο διατυπώνουμε τα συμπεράσματα της όλης διαδικασίας. Τόσο το θεωρητικό όσο και το πειραματικό μέρος αυτής της εργασίας καλύπτουν σε ένα μεγάλο ποσοστό τα βασικά στάδια ανακατασκευής του τρισδιάστατου χώρου. Τα αποτελέσματα της πειραματικής διαδικασίας αποδεικνύουν ότι οι υπάρχουσες μέθοδοι λειτουργούν ικανοποιητικά αλλά υπάρχουν πολλά περιθώρια βελτίωσης στο θέμα της Υπολογιστικής Όρασης. Στο σημείο αυτό να ευχαριστήσουμε τον επιβλέποντα καθηγητή μας κ. Δερματά για τη συνεργασία του και την κατανόησή του. / The current thesis has been written as part of the undergraduate studies for the department of Electrical and Computer Engineering of Patras University. Its objective is the three-dimensional (3D) reconstruction from two, at least, photographs, which is part of computer vision. More specifically, this thesis analyzes in detail the case of stereo vision when the camera, among two successive shots of the same image, has zero relative rotation compared to its initial position and an average translation of about 5 cm. In this way, it attempts to simulate human vision since this is essential for many Artificial Intelligence applications. Humans take stereo vision for granted since they live in a three-dimensional world. However, this world becomes two-dimensional when recorded by a camera. We can still get information about the image depth but this is empirically done based on comparing various heights, shapes and sizes. Images are identified by the computer as any other file. Computers cannot draw conclusions about what is depicted in the real world. They need to combine at least two images of the same scene and of different positions to identify the image’s depth. This process is described in the current thesis. The first chapter describes stereo vision and why it is so useful. The second chapter provides the basic principles of projective geometry, the mathematical background for passing from the two-dimensional level to the three-dimensional. The third chapter refers to camera modeling and its parameters (instrisic and extrinsic). Chapter four analyzes the camera calibration process. Chapter five explains the matching process of points of interest in both pictures. The sixth chapter provides the basic principles of epipolar geometry. The seventh chapter shows the experimental procedure that we followed in order to estimate the depth of the scene. Chapter eight shows how the 3D reconstruction is finally done. Chapter nine talks about our conclusions and how the results could improve. Both theoretical and experimental parts of this project cover the key points of 3d reconstruction. The results of the experiments show that the existing methods are satisfying but could improve more. We want to thank our supervisor professor Mr. Dermatas for his collaboration and his understanding.

Thesis (MScEng)--Stellenbosch University, 2012. / ENGLISH ABSTRACT: Simultaneous localization and mapping (SLAM) is vital for autonomous robot navigation. The robot must build a map of its environment while tracking its own motion through that map. Although many solutions to this intricate problem have been proposed, one of the most prominent issues that still needs to be resolved is to accurately measure and track landmarks over time. In this thesis we investigate the use of stereo vision for this purpose. In order to find landmarks in images we explore the use of two feature detectors: the scale-invariant feature transform (SIFT) and speeded-up robust features (SURF). Both these algorithms find salient points in images and calculate a descriptor for each point that is invariant to scale, rotation and illumination. By using the descriptors we match these image features between stereo images and use the geometry of the system to calculate a set of 3D landmark measurements. A Taylor approximation of this transformation is used to derive a Gaussian noise model for the measurements. The measured landmarks are matched to landmarks in a map to find correspondences. We find that this process often incorrectly matches ambiguous landmarks. To find these mismatches we develop a novel outlier detection scheme based on the random sample consensus (RANSAC) framework. We use a similarity transformation for the RANSAC model and derive a probabilistic consensus measure that takes the uncertainties of landmark locations into account. Through simulation and practical tests we find that this method is a significant improvement on the standard approach of using the fundamental matrix. With accurately identified landmarks we are able to perform SLAM. We investigate the use of three popular SLAM algorithms: EKF SLAM, FastSLAM and FastSLAM 2. EKF SLAM uses a Gaussian distribution to describe the systems states and linearizes the motion and measurement equations with Taylor approximations. The two FastSLAM algorithms are based on the Rao-Blackwellized particle filter that uses particles to describe the robot states, and EKFs to estimate the landmark states. FastSLAM 2 uses a refinement process to decrease the size of the proposal distribution and in doing so decreases the number of particles needed for accurate SLAM. We test the three SLAM algorithms extensively in a simulation environment and find that all three are capable of very accurate results under the right circumstances. EKF SLAM displays extreme sensitivity to landmark mismatches. FastSLAM, on the other hand, is considerably more robust against landmark mismatches but is unable to describe the six-dimensional state vector required for 3D SLAM. FastSLAM 2 offers a good compromise between efficiency and accuracy, and performs well overall. In order to evaluate the complete system we test it with real world data. We find that our outlier detection algorithm is very effective and greatly increases the accuracy of the SLAM systems. We compare results obtained by all three SLAM systems, with both feature detection algorithms, against DGPS ground truth data and achieve accuracies comparable to other state-of-the-art systems. From our results we conclude that stereo vision is viable as a sensor for SLAM. / AFRIKAANSE OPSOMMING: Gelyktydige lokalisering en kartering (simultaneous localization and mapping, SLAM) is ’n noodsaaklike proses in outomatiese robot-navigasie. Die robot moet ’n kaart bou van sy omgewing en tegelykertyd sy eie beweging deur die kaart bepaal. Alhoewel daar baie oplossings vir hierdie ingewikkelde probleem bestaan, moet een belangrike saak nog opgelos word, naamlik om landmerke met verloop van tyd akkuraat op te spoor en te meet. In hierdie tesis ondersoek ons die moontlikheid om stereo-visie vir hierdie doel te gebruik. Ons ondersoek die gebruik van twee beeldkenmerk-onttrekkers: scale-invariant feature transform (SIFT) en speeded-up robust features (SURF). Altwee algoritmes vind toepaslike punte in beelde en bereken ’n beskrywer vir elke punt wat onveranderlik is ten opsigte van skaal, rotasie en beligting. Deur die beskrywer te gebruik, kan ons ooreenstemmende beeldkenmerke soek en die geometrie van die stelsel gebruik om ’n stel driedimensionele landmerkmetings te bereken. Ons gebruik ’n Taylor- benadering van hierdie transformasie om ’n Gaussiese ruis-model vir die metings te herlei. Die gemete landmerke se beskrywers word dan vergelyk met dié van landmerke in ’n kaart om ooreenkomste te vind. Hierdie proses maak egter dikwels foute. Om die foutiewe ooreenkomste op te spoor het ons ’n nuwe uitskieterherkenningsalgoritme ontwikkel wat gebaseer is op die RANSAC-raamwerk. Ons gebruik ’n gelykvormigheidstransformasie vir die RANSAC-model en lei ’n konsensusmate af wat die onsekerhede van die ligging van landmerke in ag neem. Met simulasie en praktiese toetse stel ons vas dat die metode ’n beduidende verbetering op die standaardprosedure, waar die fundamentele matriks gebruik word, is. Met ons akkuraat geïdentifiseerde landmerke kan ons dan SLAM uitvoer. Ons ondersoek die gebruik van drie SLAM-algoritmes: EKF SLAM, FastSLAM en FastSLAM 2. EKF SLAM gebruik ’n Gaussiese verspreiding om die stelseltoestande te beskryf en Taylor-benaderings om die bewegings- en meetvergelykings te lineariseer. Die twee FastSLAM-algoritmes is gebaseer op die Rao-Blackwell partikelfilter wat partikels gebruik om robottoestande te beskryf en EKF’s om die landmerktoestande af te skat. FastSLAM 2 gebruik ’n verfyningsproses om die grootte van die voorstelverspreiding te verminder en dus die aantal partikels wat vir akkurate SLAM benodig word, te verminder. Ons toets die drie SLAM-algoritmes deeglik in ’n simulasie-omgewing en vind dat al drie onder die regte omstandighede akkurate resultate kan behaal. EKF SLAM is egter baie sensitief vir foutiewe landmerkooreenkomste. FastSLAM is meer bestand daarteen, maar kan nie die sesdimensionele verspreiding wat vir 3D SLAM vereis word, beskryf nie. FastSLAM 2 bied ’n goeie kompromie tussen effektiwiteit en akkuraatheid, en presteer oor die algemeen goed. Ons toets die hele stelsel met werklike data om dit te evalueer, en vind dat ons uitskieterherkenningsalgoritme baie effektief is en die akkuraatheid van die SLAM-stelsels beduidend verbeter. Ons vergelyk resultate van die drie SLAM-stelsels met onafhanklike DGPS-data, wat as korrek beskou kan word, en behaal akkuraatheid wat vergelykbaar is met ander toonaangewende stelsels. Ons resultate lei tot die gevolgtrekking dat stereo-visie ’n lewensvatbare sensor vir SLAM is.

The construction of fish passes has been a longstanding measure to improve river ecosystem status by ensuring the passability of weirs, dams and other in- channel structures for migratory fish. Many fish passes have a low biological effectiveness because of unsuitable hydrodynamic conditions hindering fish to rapidly detect the pass entrance. There has been a need for techniques to quantify the hydrodynamics surrounding fish pass entrances in order to identify those passes that require enhancement and to improve the design of new passes. This PhD thesis presents the development of a methodology for the rapid, spatially continuous quantification of near-pass hydrodynamics in the field. The methodology involves moving-vessel Acoustic Doppler Current Profiler (ADCP) measurements in order to quantify the 3-dimensional water velocity distribution around fish pass entrances. The approach presented in this thesis is novel because it integrates a set of techniques to make ADCP data robust against errors associated with the environmental conditions near engineered in-channel structures. These techniques provide solutions to (i) ADCP compass errors from magnetic interference, (ii) bias in water velocity data caused by spatial flow heterogeneity, (iii) the accurate ADCP positioning in locales with constrained line of sight to navigation satellites, and (iv) the accurate and cost-effective sensor deployment following pre-defined sampling strategies. The effectiveness and transferability of the methodology were evaluated at three fish pass sites covering conditions of low, medium and high discharge. The methodology outputs enabled a detailed quantitative characterisation of the fish pass attraction flow and its interaction with other hydrodynamic features. The outputs are suitable to formulate novel indicators of hydrodynamic fish pass attractiveness and they revealed the need to refine traditional fish pass design guidelines.

Articulated Heavy Goods Vehicles (HGVs) are more efficient than conventional rigid lorries, but exhibit reduced low-speed manoeuvrability and high-speed stability. Technologies such as autonomous reversing and path-following trailer steering can mitigate this, but practical limitations of the available sensing technologies restrict their commercialisation potential. This dissertation describes the development of practical vision-based articulation angle and trailer off-tracking sensing for HGVs. Chapter 1 provides a background and literature review, covering important vehicle technologies, existing commercial and experimental sensors for articulation angle and off-tracking measurement, and relevant vision-based technologies. This is followed by an introduction to pertinent computer vision theory and terminology in Chapter 2. Chapter 3 describes the development and simulation-based assessment of an articulation angle sensing concept. It utilises a rear-facing camera mounted behind the truck or tractor, and one of two proposed image processing methods: template-matching and Parallel Tracking and Mapping (PTAM). The PTAM-based method was shown to be the more accurate and versatile method in full-scale vehicle tests. RMS measurement errors of 0.4-1.6° were observed in tests on a tractor semi-trailer (Chapter 4), and 0.8-2.4° in tests on a Nordic combination with two articulation points (Chapter 5). The system requires no truck-trailer communication links or artificial markers, and is compatible with multiple trailer shapes, but was found to have increasing errors at higher articulation angles. Chapter 6 describes the development and simulation-based assessment of a trailer off-tracking sensing concept, which utilises a trailer-mounted stereo camera pair and visual odometry. The concept was evaluated in full-scale tests on a tractor semi-trailer combination in which camera location and stereo baseline were varied, presented in Chapter 7. RMS measurement errors of 0.11-0.13 m were obtained in some tests, but a sensitivity to camera alignment was discovered in others which negatively affected results. A very stiff stereo camera mount with a sub-0.5 m baseline is suggested for future experiments. A summary of the main conclusions, a review of the objectives, and recommendations for future work are given in Chapter 8. Recommendations include further refinement of both sensors, an investigation into lighting sensitivity, and alternative applications of the sensors.

Les champs de lumière ont attisé la curiosité durant ces dernières décennies. Capturés par une caméra plénoptique ou un ensemble de caméras, ils échantillonnent la fonction plénoptique qui informe sur la radiance de n'importe quel rayon lumineux traversant la scène observée. Les champs lumineux offrent de nombreuses applications en vision par ordinateur comme en infographie, de la reconstruction 3D à la segmentation, en passant par la synthèse de vue, l'inpainting ou encore le matting par exemple.Dans ce travail nous nous attelons au problème de reconstruction du champ de lumière dans le but de synthétiser une image, comme si elle avait été prise par une caméra plus proche du sujet de la scène que l'appareil de capture plénoptique. Notre approche consiste à formuler la reconstruction du champ lumineux comme un problème de rendu basé image (IBR). La plupart des algorithmes de rendu basé image s'appuient dans un premier temps sur une reconstruction 3D approximative de la scène, appelée proxy géométrique, afin d'établir des correspondances entre les points image des vues sources et ceux de la vue cible. Une nouvelle vue est générée par l'utilisation conjointe des images sources et du proxy géométrique, bien souvent par la projection des images sources sur le point de vue cible et leur fusion en intensité.Un simple mélange des couleurs des images sources ne garantit pas la cohérence de l'image synthétisée. Nous proposons donc une méthode de rendu direct multi-échelles basée sur les pyramides de laplaciens afin de fusionner les images sources à toutes les fréquences, prévenant ainsi l'apparition d'artefacts de rendu.Mais l'imperfection du proxy géométrique est aussi la cause d'artefacts de rendu, qui se traduisent par du bruit en haute fréquence dans l'image synthétisée. Nous introduisons une nouvelle méthode de rendu variationnelle avec des contraintes sur les gradients de l'image cible dans le but de mieux conditionner le système d'équation linéaire à résoudre et supprimer les artefacts de rendu dus au proxy.Certaines scènes posent de grandes difficultés de reconstruction du fait du caractère non-lambertien éventuel de certaines surfaces~; d'autre part même un bon proxy ne suffit pas, lorsque des réflexions, transparences et spécularités remettent en cause les règles de la parallaxe. Nous proposons méthode originale basée sur l'approximation locale de l'espace plénoptique à partir d'un échantillonnage épars afin de synthétiser n'importe quel point de vue sans avoir recours à la reconstruction explicite d'un proxy géométrique. Nous évaluons notre méthode à la fois qualitativement et quantitativement sur des scènes non-triviales contenant des matériaux non-lambertiens.Enfin nous ouvrons une discussion sur le problème du placement optimal de caméras contraintes pour le rendu basé image, et sur l'utilisation de nos algorithmes pour la vision d'objets dissimulés derrière des camouflages.Les différents algorithmes proposés sont illustrés par des résultats sur des jeux de données plénoptiques structurés (de type grilles de caméras) ou non-structurés. / Light fields have gathered much interest during the past few years. Captured from a plenoptic camera or a camera array, they sample the plenoptic function that provides rich information about the radiance of any ray passing through the observed scene. They offer a pletora of computer vision and graphics applications: 3D reconstruction, segmentation, novel view synthesis, inpainting or matting for instance.Reconstructing the light field consists in recovering the missing rays given the captured samples. In this work we cope with the problem of reconstructing the light field in order to synthesize an image, as if it was taken by a camera closer to the scene than the input plenoptic device or set of cameras. Our approach is to formulate the light field reconstruction challenge as an image-based rendering (IBR) problem. Most of IBR algorithms first estimate the geometry of the scene, known as a geometric proxy, to make correspondences between the input views and the target view. A new image is generated by the joint use of both the input images and the geometric proxy, often projecting the input images on the target point of view and blending them in intensity.A naive color blending of the input images do not guaranty the coherence of the synthesized image. Therefore we propose a direct multi-scale approach based on Laplacian rendering to blend the source images at all the frequencies, thus preventing rendering artifacts.However, the imperfection of the geometric proxy is also a main cause of rendering artifacts, that are displayed as a high-frequency noise in the synthesized image. We introduce a novel variational rendering method with gradient constraints on the target image for a better-conditioned linear system to solve, removing the high-frequency noise due to the geometric proxy.Some scene reconstructions are very challenging because of the presence of non-Lambertian materials; moreover, even a perfect geometric proxy is not sufficient when reflections, transparencies and specularities question the rules of parallax. We propose an original method based on the local approximation of the sparse light field in the plenoptic space to generate a new viewpoint without the need for any explicit geometric proxy reconstruction. We evaluate our method both quantitatively and qualitatively on non-trivial scenes that contain non-Lambertian surfaces.Lastly we discuss the question of the optimal placement of constrained cameras for IBR, and the use of our algorithms to recover objects that are hidden behind a camouflage.The proposed algorithms are illustrated by results on both structured (camera arrays) and unstructured plenoptic datasets.

Este trabalho aborda a utilização de tecnologias para a criação de ambientes virtuais baseados em imagens, com uma aplicação na praça central da cidade de Pelotas/RS. Possibilita que o usuário realize interativamente um passeio no entorno da praça com recursos que permitem avaliações arquitetônicas e urbanísticas por meio de simulações de troca de cor nos prédios, inserção de novas construções e análise do estado de conservação de imóveis, com vantagens sobre métodos tradicionais de análises em montagens fotográficas estáticas. Enfatiza a pesquisa de soluções para visualização interativa de ambientes, focado na identificação de fontes de informação e não na sua modelagem matemática. Para a apresentação do trabalho, além da seleção das cenas urbanas que seriam estudadas, foram consideradas técnicas para a modelagem dos elementos arquitetônicos de forma que se obtivesse a máxima preservação das características naturais (estado de conservação, cores e riqueza de detalhes), formas para a visualização destes dados de maneira integrada com dados bibliográficos (histórico, técnicas construtivas e localização) e também a utilização de objetos multimídia para o enriquecimento na apresentação de conteúdos. Na busca de soluções, foram investigadas as técnicas de modelagem fotogramétrica como forma de obtenção de objetos realísticos com a manutenção das características do objeto original em modelos geométricos simplificados. Foram feitos estudos sobre as formas de apresentação de ambientes virtuais navegáveis através de panoramas interativos e também foi estudada uma forma de aproximar a visualização dos conteúdos com a visão natural humana, através da estereoscopia. O trabalho culmina com a fusão de todas as tecnologias estudadas em uma aplicação denominada “Praça República Virtual – PRV” (para resgatar o antigo nome da atual Praça Coronel Pedro Osório), que funciona como um canteiro de provas na medida que abre um leque de opções para utilização em estudos na área de Arquitetura e Urbanismo. / This work approaches the uses of technologies for creation of virtual environments based on images, with an application to the central square of the city of Pelotas/RS. It makes possible to the user to accomplish an interactive walkthrough with resources that allows architectural and urban planning evaluations through simulations of color changes in the buildings, insertion of new constructions and analysis of the conservation condition of the buildings, with advantages if compared to traditional methods of analysis with static photographic assemblies. It emphasizes the research of solutions for interactive visualization of environments, focused in the identification of sources of information and not in mathematical modeling. Besides the selection of the urban scenes to be studied, it was considered techniques for modeling the architectural elements to obtain the maximal preservation of their natural characteristics (conservation state, colors and wealth of details), ways of visualization of nowadays conservation, but integrated with bibliographical dates (historical, constructive techniques and location). It also includes the uses of multimedia objects to the presentation of contents. In the search of solutions, techniques of photogrammetric modeling were investigated, looking for the construction of realistic objects and maintenance of the characteristics of the original object but using simplified geometric models. It was done a search about forms of presentation of navigable virtual environments through interactive panoramas and also using a prototype of stereoscopy. The work culminates with the coalition of all studied technologies in a prototype of the central square of Pelotas called “Praça República Virtual–PRV” (to rescue the old name of the current Square Cel. Pedro Osório). It works like a stonemason of proofs in the measure that opens various options of use in studies in the area of Architecture and Urban Planning.