The main purpose of this work is to investigate the behavior of the recently released by Microsoft company the Kinect sensor, which contains the properties that go beyond ordinary cameras. Normally, in order to create a 3D reconstruction of the scene two cameras are required. Whereas, the Kinect device, due to the properties of the Infrared projector and sensor allows to create the same type of the reconstruction using only one device. However, the depth images, which are generated by the Infrared laser projector and monochrome sensor in Kinect can contain undefined values. Therefore, in addition to other investigations this project contains an idea how to improve the quality of the depth images. However, the base aim of this work is to perform a reconstruction of the scene based on the color images using pair of Kinects which will be compared with the results generated by using depth information from one Kinect. In addition, the report contains the information how to check that all the performed calculations were done correctly. All  the algorithms which were used in the project as well as the achieved results will be described and discussed in the separate chapters in the current report.

L'étude de l'apparence de surfaces par analyse photométrique est un domaine de recherche actif, avec de nombreuses applications par exemple pour étudier de la qualité de surfaces, la rugosité des objets, leur apparence, etc. Le sujet de cette thèse concerne plus particulièrement l'étude de surfaces opaques, par l'acquisition de la géométrie et de la réflectance. Cela nous a conduit à une analyse des modèles mathématique de réflectance, permettant de représenter les matériaux. Afin d'offrir une description physiquement plausible des matériaux opaques, notre première contribution principale concerne la mise en oeuvre d'un modèle à base de microfacettes Lambertiennes interfacées. Il généralise différents modèles de la littérature incluant des surfaces planes diffuses ou spéculaires et rugueuses diffuses ou spéculaires grâce à trois paramètres physiques : couleur, rugosité et indice de réfraction. Il permet de prendre en compte la transmission des flux lumineux pénétrant sous la surface ainsi que les réflexions multiples entre microfacettes et de restituer les effets de rétrodiffusion lumineuse et d’anisotropie. Notre seconde contribution principale concerne la réalisation d'un système complet d'acquisition de la géométrie et de la réflectance d'objets à partir d'images HDR. Notre méthodologie correspond à une chaîne de reconstruction complète et automatique, uniquement à partir d'images, permettant d'obtenir un niveau de précision intéressant et un faible coût de mise en place et de temps de traitement comparé aux méthodes existantes. Notre méthode permet d'extraire des échantillons de réflectance suffisamment nombreux pour identifier les paramètres de modèles de réflectance avec les données acquises. / The study of surface appearance by photometric analysis is an active area of research, with various applications concerning the analysis of surface roughness or appearance ... The subject of this PhD dissertation relates to the study of opaque surfaces, through the acquisition of their geometry. Our study leads us to an analysis of mathematical reflectance models, for representing materials appearance. To provide a physically plausible description of opaque surfaces, the first major contribution concerns the implementation of a model based on Lambertian interfaced microfacets. This model generalizes several approaches often referenced in the literature, and includes flat diffuse or specular surfaces as well as diffuse or specular microfacets with three physically-based parameters: color, roughness and refractive index. It makes it possible to take into account the transmission of the light flux entering below the surface as well as multiple reflections between microfacets, while handling backscattering and anisotropy. The second main contribution of this work concerns the impolementation of a complete acquisition system for estimating geometry and reflectance from HDR images. Our methodology is based on a complete and automatic reconstruction framework, achieving a higher level of precision, a lower cost of implementation and a shorter processing time compared to photometry-based existing methods.

Ce travail décrit certains aspects du réseau d’assainissement en tant que réacteur bio-physico-chimique. Les objectifs principaux sont : d’identifier et de quantifier les sources des eaux circulant dans les collecteurs en se basant sur les signatures isotopiques des eaux et des sulfates dissous; de déterminer la nature minéralogique des phases porteuses des métaux lourds et d’étudier leur cycle géochimique par MET et MEB couplés à la EDXS ; de mettre en évidence la présence d’échange de matière entre les eaux usées et les biofilms par CLSM et MET et d’étudier les conditions hydrodynamiques impliquées dans cet échange dans un réacteur pilote (Couette-Poiseuille). On montre que les données isotopiques permettent d’évaluer la présence des évolutions de concentration des éléments transportés dans le réseau. Le rôle du réseau en tant que réacteur apparaît dans l’évolution de la nature minéralogique des phases phosphatées vers l’aval du réseau et dans la précipitation des métaux lourds dans les compartiments anaérobiques sous forme de phases sulfurées néoformées. Les dépôts des regards de façade et les biofilms sont les lieux de cette néoformation. A l’échelle de la matière organique, l’auto-épuration est révélée par les échanges entre les biofilms et les matières en suspension (MES) suite à la présence des fibres de cellulose dans la structure des biofilms et de morceaux de biofilms au sein de la MES. La majeure partie de ces évolutions se situe en amont du réseau où les contrastes physicochimiques sont les plus importants. Finalement, les expériences modèles au sein du réacteur ont montré l’implication de l’hydrodynamisme dans le détachement des biofilms / This work describes the sewer system as an integrated part of the wastewater treatment system. The main objectives of this study were: identification and quantification of water sources collected in sewer system, referring to isotopic signatures of water and dissolved sulfates; identification of the mineralogical nature of trace element carriers and determination of their geochemical evolution within the sewer by TEM and SEM coupled with EDSX ; Evidencing exchanges between sewage and biofilms, using CLSM and TEM, and investigating hydrodynamic conditions controlling this exchange in an experimental set-up (Couette-Poiseuille reactor). The results indicate that isotopic data may be used to study the tightness of sewer lines and to evaluate the evolution of element concentrations along sewer. Implication of the sewer system as a true biophysicochemical reactor is evidenced in our study by the evolution of the mineralogical nature of phosphate phases downstream of the sewer and by heavy metal precipitations in anaerobic conditions as neoformed sulfide phases. Sump pit deposits and biofilms represent the earlier stage of this neoformation. Organic matter biodegradation was revealed by TEM examination of SM whereas the exchange between biofilms and SM was shown by CLSM. Cellulose fibers from SM were found embedded in exopolymer biofilm matrices and detached fragments from biofilms were identified in sewage. The majority of these evolutions are located upstream of sewer system in which the contrast in physicochemical properties are the most significant. Finally, biofilm model investigations and image processing showed that hydrodynamic conditions are largely implicated in biofilm detachment

Virtual camera realisation and the proposition of trapezoidal camera architecture are the two broad contributions of this thesis. Firstly, multiple camera and their arrangement constitute a critical component which affect the integrity of visual content acquisition for multi-view video. Currently, linear, convergence, and divergence arrays are the prominent camera topologies adopted. However, the large number of cameras required and their synchronisation are two of prominent challenges usually encountered. The use of virtual cameras can significantly reduce the number of physical cameras used with respect to any of the known camera structures, hence adequately reducing some of the other implementation issues. This thesis explores to use image-based rendering with and without geometry in the implementations leading to the realisation of virtual cameras. The virtual camera implementation was carried out from the perspective of depth map (geometry) and use of multiple image samples (no geometry). Prior to the virtual camera realisation, the generation of depth map was investigated using region match measures widely known for solving image point correspondence problem. The constructed depth maps have been compare with the ones generated using the dynamic programming approach. In both the geometry and no geometry approaches, the virtual cameras lead to the rendering of views from a textured depth map, construction of 3D panoramic image of a scene by stitching multiple image samples and performing superposition on them, and computation of virtual scene from a stereo pair of panoramic images. The quality of these rendered images were assessed through the use of either objective or subjective analysis in Imatest software. Further more, metric reconstruction of a scene was performed by re-projection of the pixel points from multiple image samples with a single centre of projection. This was done using sparse bundle adjustment algorithm. The statistical summary obtained after the application of this algorithm provides a gauge for the efficiency of the optimisation step. The optimised data was then visualised in Meshlab software environment, hence providing the reconstructed scene. Secondly, with any of the well-established camera arrangements, all cameras are usually constrained to the same horizontal plane. Therefore, occlusion becomes an extremely challenging problem, and a robust camera set-up is required in order to resolve strongly the hidden part of any scene objects. To adequately meet the visibility condition for scene objects and given that occlusion of the same scene objects can occur, a multi-plane camera structure is highly desirable. Therefore, this thesis also explore trapezoidal camera structure for image acquisition. The approach here is to assess the feasibility and potential of several physical cameras of the same model being sparsely arranged on the edge of an efficient trapezoid graph. This is implemented both Matlab and Maya. The quality of the depth maps rendered in Matlab are better in Quality.

Bayesian methods provide a principled approach to some of the hardest problems in computer vision—low signal-to-noise ratios, ill-posed problems, and problems with missing data. This dissertation applies Bayesian modeling to infer multidimensional continuous manifolds (e.g., curves, surfaces) from image data using Gaussian process priors. Gaussian processes are ideal priors in this setting, providing a stochastic model over continuous functions while permitting efficient inference. We begin by introducing a formal mathematical representation of branch curvilinear structures called a curve tree and we define a novel family of Gaussian processes over curve trees called branching Gaussian processes. We define two types of branching Gaussian properties and show how to extend them to branching surfaces and hypersurfaces. We then apply Gaussian processes in three computer vision applications. First, we perform 3D reconstruction of moving plants from 2D images. Using a branching Gaussian process prior, we recover high quality 3D trees while being robust to plant motion and camera calibration error. Second, we perform multi-part segmentation of plant leaves from highly occluded silhouettes using a novel Gaussian process model for stochastic shape. Our method obtains good segmentations despite highly ambiguous shape evidence and minimal training data. Finally, we estimate 2D trees from microscope images of neurons with highly ambiguous branching structure. We first fit a tree to a blurred version of the image where structure is less ambiguous. Then we iteratively deform and expand the tree to fit finer images, using a branching Gaussian process regularizing prior for deformation. Our method infers natural tree topologies despite ambiguous branching and image data containing loops. Our work shows that Gaussian processes can be a powerful building block for modeling complex structure, and they perform well in computer vision problems having significant noise and ambiguity.

Modèle 3D d'objet, séquences vidéos monoscopiques, estimation de la pose 3D, recalage 3D/2D, texture, flot optique, translation et rotation de grande amplitude, occultation, appariement par bloc, interpolation temporelle, modélisation ondulatoire, critère de visibilité, analyse de déformations faciales, description MPEG-4 du visage, prototype déformable, bouche, yeux, B-splines, classification floue non supervisée, méthode du simplexe, synthèse de déformations faciales..

La production de dessins animés 2D, qui suit actuellement un schéma mis en place dans les années 1920, fait intervenir un très nombre de compétences humaines et de métiers différents. Par opposition à ce mode de travail traditionnel, la production de films de synthèse 3D, en exploitant les technologies et outils les plus récents de modélisation et d'animation 3D, s'affranchit pour une bonne part de cette composante artisanale et vient concurrencer l'industrie du dessin animé tradtionnel en termes de délais et de coûts de fabrication.<br /><br />Les défis à relever par l'industrie du dessin animé 2D se formulent donc en termes de:<br /><br /> 1. Réutilisation des contenus selon le paradigme Create once, render many,<br /> 2. Facilité d'échange et de transmission des contenus, ce qui nécessite de disposer d'un unique format de représentation,<br /> 3. Production efficace et économique des contenus, requérant une animation automatisée par ordinateur. <br /><br />Dans ce contexte compétitif, ce travail de thèse, réalisé dans le cadre du projet industriel TOON financé par la société Quadraxis avec le support de l' Agence Nationale de Valorisation de la Recherche (Oséo-ANVAR), a pour objectif de contribuer au développement d'une plate-forme de reconstruction, déformation et animation de modèles 3D pour les dessins animés 2D.<br /><br />Un état de l'art des méthodes et outils contribuant à la reconstruction de modèles 3D et à leur animation est présenté et discuté au regard des contraintes spécifiques des règles de création des dessins animés 2D et de la chaîne de fabrication traditionnelle. Ayant identifié les verrous technologiques à lever, nos contributions ont porté sur :<br /><br /> * l'élaboration d'une méthode de reconstruction de personnages virtuels 3D à partir de dessins 2D,<br /> * la mise au point d'une procédure de reconstruction surfacique par NURBS dotée d'une capacité de déformation interactive 2D/3D,<br /> * la conception d'un module de modélisation 3D pour surfaces maillées, compatible avec le standard d'animation MPEG-4/AFX. <br /><br />Les développements réalisés, intégrés dans un prototype de la plate-forme TOON, montrent un gain en temps de 20% sur l'ensemble de la chaîne de production tout en garantissant une complète interopérabilité des applications via le standard MPEG-4.

In recent years there has been significant progress in increasing the scope, accuracy and flexibility of 3D photography methods. However there are still significant open problems where complex optical properties of mirroring or transparent objects cause many assumptions of traditional algorithms to break down. In this work we present three approaches that attempt to deal with some of these challenges using a few camera views and simple illumination. First, we consider the problem of reconstructing the 3D position and surface normal of points on a time-varying refractive surface. We show that two viewpoints are sufficient to solve this problem in the general case, even if the refractive index is unknown. We introduce a novel ``stereo matching'' criterion called refractive disparity, appropriate for refractive scenes, and develop an optimization-based algorithm for individually reconstructing the position and normal of each point projecting to a pixel in the input views. Second, we present a new method for reconstructing the exterior surface of a complex transparent scene with inhomogeneous interior. We capture images from each viewpoint while moving a proximal light source to a 2D or 3D set of positions giving a 2D (or 3D) dataset per pixel, called the scatter-trace. The key is that while light transport within a transparent scene's interior can be exceedingly complex, a pixel's scatter trace has a highly-constrained geometry that reveals the direct surface reflection, and leads to a simple ``Scatter-trace stereo'' algorithm for computing the exterior surface geometry. Finally, we develop a reconstruction system for scenes with reflectance properties ranging from diffuse to specular. We capture images of the scene as it is illuminated by a planar, spatially non-uniform light source. Then we show that if the source is translated to a parallel position away from the scene, a particular scene point integrates a magnified region of light from the plane. We observe this magnification at each pixel and show how it relates to the source-relative depth of the surface. Next we show how calibration relating the camera and source planes allows for robustness to specular objects and recovery of 3D surface points.

Male crickets produce a species specific song to attract females which in response move towards the sound source. This behaviour, termed phonotaxis, has been the subject of many morphological, neurophysiological and behavioural studies making it one of the most well studied examples of acoustic communication in the animal kingdom. Despite this fact, the precise leg movements during this behaviour is unknown. This is of specific interest as the cricket’s ears are located on their front legs, meaning that the perception of the sound input might change as the insect moves. This dissertation describes a methodology and an analysis that fills this knowledge gap. I developed a semi-automated tracking system for insect motion based on commercially available high-speed video cameras and freely available software. I used it to collect detailed three dimensional kinematic information from female crickets performing free walking phonotaxis towards a calling song stimulus. I marked the insect’s joints with small dots of paint and recorded the movements from underneath with a pair of cameras following the insect as it walks on the transparent floor of an arena. Tracking is done offline, utilizing a kinematic model to constrain the processing. I obtained, for the first time, the positions and angles of all joints of all legs and six additional body joints, synchronised with stance-swing transitions and the sound pattern, at a 300 Hz frame rate. I then analysed this data based on four categories: The single leg motion analysis revealed the importance of the thoraco-coxal (ThC) and body joints in the movement of the insect. Furthermore the inside middle leg’s tibio-tarsal (TiTa) joint was the centre of the rotation during turning. Certain joints appear to be the most crucial ones for the transition from straight walking to turning. The leg coordination analysis revealed the patterns followed during straight walking and turning. Furthermore, some leg combinations cannot be explained by current coordination rules. The angles relative to the active speaker revealed the deviation of the crickets as they followed a meandering course towards it. The estimation of ears’ input revealed the differences between the two sides as the insect performed phonotaxis by using a simple algorithm. In general, the results reveal both similarities and differences with other cricket studies and other insects such as cockroaches and stick insects. The work presented herein advances the current knowledge on cricket phonotactic behaviour and will be used in the further development of models of neural control of phonotaxis.

Mon travail de thèse consiste en la mise en place d'une méthode de reconstruction 3D. Les objets à reconstruire sont des petits corps du système solaire qui ont été observée dans le domaine visible par des sondes lors de missions spatiales. La méthode proposée (appelée photoclinométrie par déformation) consiste à déformer un maillage jusqu'à ce que les images synthétiques de ce maillage correspondent aux images observées, ceci s'effectue au sein d'une boucle d'optimisation. Cette méthode nécessitant un maillage de départ proche de la solution souhaitée, nous avons implémenter cette méthode au sein d'un schéma multirésolution (multirésolution photoclinométrie par déformation) permettant d'obtenir une reconstruction sans apriori de forme. Cependant cette méthode ne permet pas d'obtenir de modèles contenant des millions de facettes. Une troisième méthode a donc été développé pour pallier ce problème, elle consiste à découper le maillage en différents morceaux et à appliquer la méthode multirésolution photoclinométrie par déformation à chacun de ces morceaux. Par fusion de ces morceaux, on arrive ainsi à obtenir des maillages de plusieurs millions de facettes (modèle haute résolution). Ces méthodes ont été testées avec la reconstruction des deux astéroïdes (Steins and lutetia) survolé par la sonde Rosetta de l'Agence Spatiale Européenne (ESA). / My PhD work consists in the establishment of a method of 3D reconstruction. Studied objects are small solar system bodies that have been observed in the visible range by sensors during space missions. The proposed method (called photoclinometry by deformation ) consists in deforming a mesh until the synthetic images of the mesh correspond to the observed ones, this is done in an optimization loop . This method requires an initial mesh close to the desired solution. We implement this method in a multiresolution scheme (multiresolution photoclinometry by deformation) to obtain a reconstruction without an apriori shape . However, this method does not allow models containing millions of facets . A third method has been developed to overcome this problem, it consists in cutting the mesh into different pieces and apply the multiresolution method photoclinometry by deforming to each of these pieces . By merging these pieces , we obtain meshes of several millions facets (high resolution model). These methods have been tested by reconstructing the two asteroids (Steins and lutetia) flown-by the Rosetta spacecraft of the European Space Agency (ESA).