This dissertation proposes an approach for 3D micro-scale shape reconstruction using photometric stereo (PS) with surface normal integration (SNI). Based on the proposed approach, a portable cost-effective stationary system is developed to capture 3D shapes in the order of micrometer scale. The PS with SNI technique is adopted to reconstruct 3D microtopology since this technique is highlighted for its capability to reproduce fine surface details at pixel resolution. Furthermore, since the primary hardware components are merely a camera and several typical LEDs, the system based on PS with SNI can be made portable at low cost. The principal contributions are three folds. First, a PS method based on dichromatic reflectance model (DRM) using color input images is proposed to generalize PS applicable to a wider range of surfaces with non-Lambertian reflectances. The proposed method not only estimates surface orientations from diffuse reflection but also exploits information from specularities owing to the proposed diffuse-specular separation algorithm. Using the proposed PS method, material-dependent features can be simultaneously extracted in addition to surface orientations, which offers much richer information in understanding the 3D scene and poses more potential functionalities, such as specular removal, intrinsic image decomposition, digital relighting, material-based segmentation, material transfer and material classification. The second contribution is the development of an SNI method dealing with perspective distortion. The proposed SNI is performed on the image plane instead of on the target surface as did by orthographic SNI owing to the newly derived representation of surface normals. The motivation behind the representation is from the observation that spatially uniform image points are simpler for integration than the non-uniform distribution of surface points under perspective projection. The new representation is then manipulated to the so-called log gradient space in analogy to the gradient space in orthographic SNI. With this analogy, the proposed method can inherit most past algorithms developed for orthographic SNI. By applying the proposed SNI, perspective distortion can be efficiently tackled with for smooth surfaces. In addition, the method is PS-independent, which can keep the image irradiance equation in a simple form during PS. The third contribution is the design and calibration of a 3D micro-scale shape reconstruction system using the derived PS and SNI methods. This system is originally designed for on-site measurement of pavement microtexture, while its applicability can be generalized to a wider range of surfaces. Optimal illumination was investigated in theory and through numerical simulations. Five different calibrations regarding various aspects of the system were either newly proposed or modified from existing methods. The performances of these calibrations were individually evaluated. Efficacy of the developed system was finally demonstrated through comprehensive comparative studies with existing systems. Its capability for on-site measurement was also confirmed. / Ph. D. / Shapes in our world are three-dimensional (3D). How to measure and digitize shapes in 3D into computer understandable virtual models using cameras is called 3D shape reconstruction in the field of computer vision. This dissertation concerns the problem of 3D shape reconstruction, while concentrates on recovering shapes at micro-meter scale, referred to as 3D micro-scale shape reconstruction. Quantifying 3D shapes at micro-scale is significant for both industry and academia. In industry, quantification of 3D shapes at micro-meter scale can be employed in precision parts manufacturing, industrial quality control and rapid prototyping, whilst in academia, even finer resolution may be required to study the microtopography of a surface, such as for the purpose of investigating the nature of friction between surfaces. In this dissertation, a systematic solution is given for 3D micro-scale shape reconstruction using techniques called photometric stereo (PS) and surface normal integration (SNI) sequentially. PS estimates surface normals for each pixel-corresponding surface patch using images captured under various illumination directions from a fixed viewpoint. These surface normals are then integrated to reconstruct the surface in 3D via SNI. Based on these general principles, a prototype system was developed. The hardware of the system is simple, mainly contains a color digital single-lens reflex (DSLR) camera with a macro lens, multiple LEDs, a control circuit and a cover. During operation, the LEDs are sequentially turned on and create different illuminations upon the surface of concern. The DSLR camera simultaneously captures images with one LED lit at a time. Having these images for the target surface under various illuminations, the 3D surface at micro-scale is reconstructed through post-processing by PS with SNI algorithms. Three principal contributions are presented in this dissertation. First, a PS algorithm using color images is demonstrated to improve the shape reconstruction accuracy and its applicability for a wider range of surfaces with different reflectance properties. The proposed PS algorithm can also estimate material-dependent properties of the surface, making potential applications, such as material classification and inference, feasible. The second contribution is to improve the SNI algorithm to deal with the camera’s perspective distortion. Experimental results suggested that the algorithm has been successful in dealing with the distortion for smooth surfaces. The design and calibration of the prototype system are presented as the third contribution. The system can achieve high data acquisition rate due to its area scanning nature, dense measurements at micro-scale due to the PS with SNI approach, and low-cost due to the simple hardware configurations. Efficacy of the system was demonstrated through comprehensive comparative studies with existing systems. Its capability for on-site measurement was also proven.

The main purpose of this work is to investigate the behavior of the recently released by Microsoft company the Kinect sensor, which contains the properties that go beyond ordinary cameras. Normally, in order to create a 3D reconstruction of the scene two cameras are required. Whereas, the Kinect device, due to the properties of the Infrared projector and sensor allows to create the same type of the reconstruction using only one device. However, the depth images, which are generated by the Infrared laser projector and monochrome sensor in Kinect can contain undefined values. Therefore, in addition to other investigations this project contains an idea how to improve the quality of the depth images. However, the base aim of this work is to perform a reconstruction of the scene based on the color images using pair of Kinects which will be compared with the results generated by using depth information from one Kinect. In addition, the report contains the information how to check that all the performed calculations were done correctly. All  the algorithms which were used in the project as well as the achieved results will be described and discussed in the separate chapters in the current report.

L'étude de l'apparence de surfaces par analyse photométrique est un domaine de recherche actif, avec de nombreuses applications par exemple pour étudier de la qualité de surfaces, la rugosité des objets, leur apparence, etc. Le sujet de cette thèse concerne plus particulièrement l'étude de surfaces opaques, par l'acquisition de la géométrie et de la réflectance. Cela nous a conduit à une analyse des modèles mathématique de réflectance, permettant de représenter les matériaux. Afin d'offrir une description physiquement plausible des matériaux opaques, notre première contribution principale concerne la mise en oeuvre d'un modèle à base de microfacettes Lambertiennes interfacées. Il généralise différents modèles de la littérature incluant des surfaces planes diffuses ou spéculaires et rugueuses diffuses ou spéculaires grâce à trois paramètres physiques : couleur, rugosité et indice de réfraction. Il permet de prendre en compte la transmission des flux lumineux pénétrant sous la surface ainsi que les réflexions multiples entre microfacettes et de restituer les effets de rétrodiffusion lumineuse et d’anisotropie. Notre seconde contribution principale concerne la réalisation d'un système complet d'acquisition de la géométrie et de la réflectance d'objets à partir d'images HDR. Notre méthodologie correspond à une chaîne de reconstruction complète et automatique, uniquement à partir d'images, permettant d'obtenir un niveau de précision intéressant et un faible coût de mise en place et de temps de traitement comparé aux méthodes existantes. Notre méthode permet d'extraire des échantillons de réflectance suffisamment nombreux pour identifier les paramètres de modèles de réflectance avec les données acquises. / The study of surface appearance by photometric analysis is an active area of research, with various applications concerning the analysis of surface roughness or appearance ... The subject of this PhD dissertation relates to the study of opaque surfaces, through the acquisition of their geometry. Our study leads us to an analysis of mathematical reflectance models, for representing materials appearance. To provide a physically plausible description of opaque surfaces, the first major contribution concerns the implementation of a model based on Lambertian interfaced microfacets. This model generalizes several approaches often referenced in the literature, and includes flat diffuse or specular surfaces as well as diffuse or specular microfacets with three physically-based parameters: color, roughness and refractive index. It makes it possible to take into account the transmission of the light flux entering below the surface as well as multiple reflections between microfacets, while handling backscattering and anisotropy. The second main contribution of this work concerns the impolementation of a complete acquisition system for estimating geometry and reflectance from HDR images. Our methodology is based on a complete and automatic reconstruction framework, achieving a higher level of precision, a lower cost of implementation and a shorter processing time compared to photometry-based existing methods.

Ce travail décrit certains aspects du réseau d’assainissement en tant que réacteur bio-physico-chimique. Les objectifs principaux sont : d’identifier et de quantifier les sources des eaux circulant dans les collecteurs en se basant sur les signatures isotopiques des eaux et des sulfates dissous; de déterminer la nature minéralogique des phases porteuses des métaux lourds et d’étudier leur cycle géochimique par MET et MEB couplés à la EDXS ; de mettre en évidence la présence d’échange de matière entre les eaux usées et les biofilms par CLSM et MET et d’étudier les conditions hydrodynamiques impliquées dans cet échange dans un réacteur pilote (Couette-Poiseuille). On montre que les données isotopiques permettent d’évaluer la présence des évolutions de concentration des éléments transportés dans le réseau. Le rôle du réseau en tant que réacteur apparaît dans l’évolution de la nature minéralogique des phases phosphatées vers l’aval du réseau et dans la précipitation des métaux lourds dans les compartiments anaérobiques sous forme de phases sulfurées néoformées. Les dépôts des regards de façade et les biofilms sont les lieux de cette néoformation. A l’échelle de la matière organique, l’auto-épuration est révélée par les échanges entre les biofilms et les matières en suspension (MES) suite à la présence des fibres de cellulose dans la structure des biofilms et de morceaux de biofilms au sein de la MES. La majeure partie de ces évolutions se situe en amont du réseau où les contrastes physicochimiques sont les plus importants. Finalement, les expériences modèles au sein du réacteur ont montré l’implication de l’hydrodynamisme dans le détachement des biofilms / This work describes the sewer system as an integrated part of the wastewater treatment system. The main objectives of this study were: identification and quantification of water sources collected in sewer system, referring to isotopic signatures of water and dissolved sulfates; identification of the mineralogical nature of trace element carriers and determination of their geochemical evolution within the sewer by TEM and SEM coupled with EDSX ; Evidencing exchanges between sewage and biofilms, using CLSM and TEM, and investigating hydrodynamic conditions controlling this exchange in an experimental set-up (Couette-Poiseuille reactor). The results indicate that isotopic data may be used to study the tightness of sewer lines and to evaluate the evolution of element concentrations along sewer. Implication of the sewer system as a true biophysicochemical reactor is evidenced in our study by the evolution of the mineralogical nature of phosphate phases downstream of the sewer and by heavy metal precipitations in anaerobic conditions as neoformed sulfide phases. Sump pit deposits and biofilms represent the earlier stage of this neoformation. Organic matter biodegradation was revealed by TEM examination of SM whereas the exchange between biofilms and SM was shown by CLSM. Cellulose fibers from SM were found embedded in exopolymer biofilm matrices and detached fragments from biofilms were identified in sewage. The majority of these evolutions are located upstream of sewer system in which the contrast in physicochemical properties are the most significant. Finally, biofilm model investigations and image processing showed that hydrodynamic conditions are largely implicated in biofilm detachment

Virtual camera realisation and the proposition of trapezoidal camera architecture are the two broad contributions of this thesis. Firstly, multiple camera and their arrangement constitute a critical component which affect the integrity of visual content acquisition for multi-view video. Currently, linear, convergence, and divergence arrays are the prominent camera topologies adopted. However, the large number of cameras required and their synchronisation are two of prominent challenges usually encountered. The use of virtual cameras can significantly reduce the number of physical cameras used with respect to any of the known camera structures, hence adequately reducing some of the other implementation issues. This thesis explores to use image-based rendering with and without geometry in the implementations leading to the realisation of virtual cameras. The virtual camera implementation was carried out from the perspective of depth map (geometry) and use of multiple image samples (no geometry). Prior to the virtual camera realisation, the generation of depth map was investigated using region match measures widely known for solving image point correspondence problem. The constructed depth maps have been compare with the ones generated using the dynamic programming approach. In both the geometry and no geometry approaches, the virtual cameras lead to the rendering of views from a textured depth map, construction of 3D panoramic image of a scene by stitching multiple image samples and performing superposition on them, and computation of virtual scene from a stereo pair of panoramic images. The quality of these rendered images were assessed through the use of either objective or subjective analysis in Imatest software. Further more, metric reconstruction of a scene was performed by re-projection of the pixel points from multiple image samples with a single centre of projection. This was done using sparse bundle adjustment algorithm. The statistical summary obtained after the application of this algorithm provides a gauge for the efficiency of the optimisation step. The optimised data was then visualised in Meshlab software environment, hence providing the reconstructed scene. Secondly, with any of the well-established camera arrangements, all cameras are usually constrained to the same horizontal plane. Therefore, occlusion becomes an extremely challenging problem, and a robust camera set-up is required in order to resolve strongly the hidden part of any scene objects. To adequately meet the visibility condition for scene objects and given that occlusion of the same scene objects can occur, a multi-plane camera structure is highly desirable. Therefore, this thesis also explore trapezoidal camera structure for image acquisition. The approach here is to assess the feasibility and potential of several physical cameras of the same model being sparsely arranged on the edge of an efficient trapezoid graph. This is implemented both Matlab and Maya. The quality of the depth maps rendered in Matlab are better in Quality.

Bayesian methods provide a principled approach to some of the hardest problems in computer vision—low signal-to-noise ratios, ill-posed problems, and problems with missing data. This dissertation applies Bayesian modeling to infer multidimensional continuous manifolds (e.g., curves, surfaces) from image data using Gaussian process priors. Gaussian processes are ideal priors in this setting, providing a stochastic model over continuous functions while permitting efficient inference. We begin by introducing a formal mathematical representation of branch curvilinear structures called a curve tree and we define a novel family of Gaussian processes over curve trees called branching Gaussian processes. We define two types of branching Gaussian properties and show how to extend them to branching surfaces and hypersurfaces. We then apply Gaussian processes in three computer vision applications. First, we perform 3D reconstruction of moving plants from 2D images. Using a branching Gaussian process prior, we recover high quality 3D trees while being robust to plant motion and camera calibration error. Second, we perform multi-part segmentation of plant leaves from highly occluded silhouettes using a novel Gaussian process model for stochastic shape. Our method obtains good segmentations despite highly ambiguous shape evidence and minimal training data. Finally, we estimate 2D trees from microscope images of neurons with highly ambiguous branching structure. We first fit a tree to a blurred version of the image where structure is less ambiguous. Then we iteratively deform and expand the tree to fit finer images, using a branching Gaussian process regularizing prior for deformation. Our method infers natural tree topologies despite ambiguous branching and image data containing loops. Our work shows that Gaussian processes can be a powerful building block for modeling complex structure, and they perform well in computer vision problems having significant noise and ambiguity.

Modèle 3D d'objet, séquences vidéos monoscopiques, estimation de la pose 3D, recalage 3D/2D, texture, flot optique, translation et rotation de grande amplitude, occultation, appariement par bloc, interpolation temporelle, modélisation ondulatoire, critère de visibilité, analyse de déformations faciales, description MPEG-4 du visage, prototype déformable, bouche, yeux, B-splines, classification floue non supervisée, méthode du simplexe, synthèse de déformations faciales..

La production de dessins animés 2D, qui suit actuellement un schéma mis en place dans les années 1920, fait intervenir un très nombre de compétences humaines et de métiers différents. Par opposition à ce mode de travail traditionnel, la production de films de synthèse 3D, en exploitant les technologies et outils les plus récents de modélisation et d'animation 3D, s'affranchit pour une bonne part de cette composante artisanale et vient concurrencer l'industrie du dessin animé tradtionnel en termes de délais et de coûts de fabrication.<br /><br />Les défis à relever par l'industrie du dessin animé 2D se formulent donc en termes de:<br /><br /> 1. Réutilisation des contenus selon le paradigme Create once, render many,<br /> 2. Facilité d'échange et de transmission des contenus, ce qui nécessite de disposer d'un unique format de représentation,<br /> 3. Production efficace et économique des contenus, requérant une animation automatisée par ordinateur. <br /><br />Dans ce contexte compétitif, ce travail de thèse, réalisé dans le cadre du projet industriel TOON financé par la société Quadraxis avec le support de l' Agence Nationale de Valorisation de la Recherche (Oséo-ANVAR), a pour objectif de contribuer au développement d'une plate-forme de reconstruction, déformation et animation de modèles 3D pour les dessins animés 2D.<br /><br />Un état de l'art des méthodes et outils contribuant à la reconstruction de modèles 3D et à leur animation est présenté et discuté au regard des contraintes spécifiques des règles de création des dessins animés 2D et de la chaîne de fabrication traditionnelle. Ayant identifié les verrous technologiques à lever, nos contributions ont porté sur :<br /><br /> * l'élaboration d'une méthode de reconstruction de personnages virtuels 3D à partir de dessins 2D,<br /> * la mise au point d'une procédure de reconstruction surfacique par NURBS dotée d'une capacité de déformation interactive 2D/3D,<br /> * la conception d'un module de modélisation 3D pour surfaces maillées, compatible avec le standard d'animation MPEG-4/AFX. <br /><br />Les développements réalisés, intégrés dans un prototype de la plate-forme TOON, montrent un gain en temps de 20% sur l'ensemble de la chaîne de production tout en garantissant une complète interopérabilité des applications via le standard MPEG-4.

In recent years there has been significant progress in increasing the scope, accuracy and flexibility of 3D photography methods. However there are still significant open problems where complex optical properties of mirroring or transparent objects cause many assumptions of traditional algorithms to break down. In this work we present three approaches that attempt to deal with some of these challenges using a few camera views and simple illumination. First, we consider the problem of reconstructing the 3D position and surface normal of points on a time-varying refractive surface. We show that two viewpoints are sufficient to solve this problem in the general case, even if the refractive index is unknown. We introduce a novel ``stereo matching'' criterion called refractive disparity, appropriate for refractive scenes, and develop an optimization-based algorithm for individually reconstructing the position and normal of each point projecting to a pixel in the input views. Second, we present a new method for reconstructing the exterior surface of a complex transparent scene with inhomogeneous interior. We capture images from each viewpoint while moving a proximal light source to a 2D or 3D set of positions giving a 2D (or 3D) dataset per pixel, called the scatter-trace. The key is that while light transport within a transparent scene's interior can be exceedingly complex, a pixel's scatter trace has a highly-constrained geometry that reveals the direct surface reflection, and leads to a simple ``Scatter-trace stereo'' algorithm for computing the exterior surface geometry. Finally, we develop a reconstruction system for scenes with reflectance properties ranging from diffuse to specular. We capture images of the scene as it is illuminated by a planar, spatially non-uniform light source. Then we show that if the source is translated to a parallel position away from the scene, a particular scene point integrates a magnified region of light from the plane. We observe this magnification at each pixel and show how it relates to the source-relative depth of the surface. Next we show how calibration relating the camera and source planes allows for robustness to specular objects and recovery of 3D surface points.

Male crickets produce a species specific song to attract females which in response move towards the sound source. This behaviour, termed phonotaxis, has been the subject of many morphological, neurophysiological and behavioural studies making it one of the most well studied examples of acoustic communication in the animal kingdom. Despite this fact, the precise leg movements during this behaviour is unknown. This is of specific interest as the cricket’s ears are located on their front legs, meaning that the perception of the sound input might change as the insect moves. This dissertation describes a methodology and an analysis that fills this knowledge gap. I developed a semi-automated tracking system for insect motion based on commercially available high-speed video cameras and freely available software. I used it to collect detailed three dimensional kinematic information from female crickets performing free walking phonotaxis towards a calling song stimulus. I marked the insect’s joints with small dots of paint and recorded the movements from underneath with a pair of cameras following the insect as it walks on the transparent floor of an arena. Tracking is done offline, utilizing a kinematic model to constrain the processing. I obtained, for the first time, the positions and angles of all joints of all legs and six additional body joints, synchronised with stance-swing transitions and the sound pattern, at a 300 Hz frame rate. I then analysed this data based on four categories: The single leg motion analysis revealed the importance of the thoraco-coxal (ThC) and body joints in the movement of the insect. Furthermore the inside middle leg’s tibio-tarsal (TiTa) joint was the centre of the rotation during turning. Certain joints appear to be the most crucial ones for the transition from straight walking to turning. The leg coordination analysis revealed the patterns followed during straight walking and turning. Furthermore, some leg combinations cannot be explained by current coordination rules. The angles relative to the active speaker revealed the deviation of the crickets as they followed a meandering course towards it. The estimation of ears’ input revealed the differences between the two sides as the insect performed phonotaxis by using a simple algorithm. In general, the results reveal both similarities and differences with other cricket studies and other insects such as cockroaches and stick insects. The work presented herein advances the current knowledge on cricket phonotactic behaviour and will be used in the further development of models of neural control of phonotaxis.