<p dir="ltr">Over the last few decades, three-dimensional (3D) optical imaging and sensing techniques have attracted much attention from both academia and industries. Owing to its capability of gathering more information than conventional 2D imaging, it has been successfully adopted in many applications on the macro scale which ranges from sub-meters to meters such as entertainment, commercial electronics, manufacturing, and construction. For example, the iPhone “FaceID” sensor is used for facial recognition, and the Microsoft Kinect is used to track body motion in video games. With recent advances in many technical fields, such as semiconductor packaging, additive manufacturing, and micro-robots, there is an increasing need for microscopic 3D imaging, and several techniques including interferometry, confocal microscopy, focus variation, and structured light have been developed and adopted in these industries. Among these techniques, the structured light 3D imaging technique is considered one of the most promising techniques for in-situ metrology, owing to its advantage of simple configuration and high measurement speed. However, several challenges must be addressed in employing the structured-light 3D imaging technique in these fields.</p><p dir="ltr">The first challenge is the limited measurement range caused by the limited depth of field (DOF). Given the necessity for large magnification in the microscopic structured light system, the DOF becomes notably shallow, especially when pin-hole lenses are adopted. This issue is exacerbated by the fact that the measured objects in the aforementioned industries could contain miniaturized features spanning a broad height range. To address this problem, we introduce the idea of the focus stacking technique, wherein the focused pixels gathered from various focus settings are merged to form an all-in-focus image, into the structured-light 3D imaging. We further developed a computational framework that utilizes the phase information and fringe contrast of the projected fringe patterns to mitigate the influence of object textures.</p><p dir="ltr">The second challenge is the 3D imaging speed. The 3D measurement speed is a crucial factor for in-situ applications. We improved the large DOF 3D imaging speed by reducing the required fringe images from two aspects: 1) We developed a calibration method for multifocus pin-hole mode, which can eliminate the necessity of the 2D image alignment. The conventional method based on circle patterns will be affected during the feature extraction process by the significant camera defocusing. In contrast, our proposed method is more robust since it uses virtual features extracted from a reconstructed white flat surface under a pre-calibrated focus setting. 2)We developed a phase unwrapping method with the assistance of the electrically tunable lens (ETL), which is an optical component we used to capture fringe images under various focus settings. The proposed phase unwrapping method leverages the focal plane position of each focus setting to estimate a rough depth map for the geometric-constraint phase unwrapping algorithm. By doing this, the method eliminates the limitation on the effective working depth range and becomes feasible in large DOF 3D imaging.</p><h4>Even with all previous methodologies, the efficiency of large DOF 3D imaging is still not high enough under certain circumstances. One of the major reasons is that we can still only use a series of pre-defined focus settings to run the focus stacking, since we have no prior on the measured objects. This issue could lead to low measurement efficiency when the depth range of the measured objects does not cover the whole enlarged DOF. To improve the performance of the system under such situations, we developed a method that introduces another computational imaging technique: the focal sweep technique, to help determine the optimal focus settings adapting to different measured objects.</h4><h4>In summary, this dissertation contributed to high-speed, large depth-of-field, and automated 3D imaging, which can be used in micro-scale applications from the following aspects: (1) enlarging the DOF of the microscopic 3D imaging using the focus stacking technique; (2) developing methods to improve the speed of large DOF microscopic 3D imaging; and (3) developing a method to improve the efficiency of the focus stacking under certain circumstances. These contributions can potentially enable the structured-light 3D imaging technique to be an alternative 3D microscopy approach for many academic studies and industry applications.</h4><p></p>

In our society with its aging population, the design and implementation of a highperformance distributed multi-sensor and information system for autonomous physical services become more and more important. In line with this, this thesis proposes an Intelligent Multi-Sensor System, IMSS, that surveys a human activities space to detect and identify a target for a specific service. The subject of this thesis covers three main aspects related to the set-up of an IMSS: an improved depth measurement and reconstruction method and its related uncertainty, a surveillance and tracking algorithm and finally a way to validate and evaluate the proposed methods and algorithms. The thesis discusses how a model of the depth spatial quantisation uncertainty can be implemented to optimize the configuration of a sensor system to capture information of the target objects and their environment with required specifications. The thesis introduces the dithering algorithm which significantly reduces the depth reconstruction uncertainty. Furthermore, the dithering algorithm is implemented on a sensor-shifted stereo camera, thus simplifying depth reconstruction without compromising the common stereo field of view. To track multiple targets continuously, the Gaussian Mixture Probability Hypothesis Density, GM-PHD, algorithm is implemented with the help of vision and Radio Frequency Identification, RFID, technologies. The performance of the tracking algorithm in a vision system is evaluated by a circular motion test signal. The thesis introduces constraints to the target space, the stereo pair characteristics and the depth reconstruction accuracy to optimize the vision system and to control the performance of surveillance and 3D reconstruction through integer linear programming. The human being within the activity space is modelled as a tetrahedron, and a field of view in spherical coordinates are used in the control algorithms. In order to integrate human behaviour and perception into a technical system, the proposed adaptive measurement method makes use of the Fuzzily Defined Variable, FDV. The FDV approach enables an estimation of the quality index based on qualitative and quantitative factors for image quality evaluation using a neural network. The thesis consists of two parts, where Part I gives an overview of the applied theory and research methods used, and Part II comprises the eight papers included in the thesis.

The three-dimensional structure of the world makes 3D as the natural evolution of a huge panel of applications. Many different 3D reconstruction algorithms have been implemented to meet different application needs. We target immersive applications. The reconstructed models of users and objects are inserted in 3D virtual environments creating the mixed reality, which is rendered on spatially immersive displays. The user sees its model on the projection walls, allowing him to interact with elements of the virtual world. Immersion requires small latency, high reconstruction rates and non-invasive systems. Furthermore, we choose to reconstruct arbitrary shapes with a geometric method. The review of the state of the art shows that all the acquisition devices cannot be connected to a single computer performing real time reconstruction. The system needs a cluster of computers and a strategy to share information between them. We present a distributed and scalable architecture for real time reconstruction of arbitrary shapes exploiting inter-frame redundancy. The architecture is composed of acquisition nodes and master nodes. Each acquisition node reconstructs partial models from its attached cameras and sends non-redundant information to its master. Each master node merges several partial models. The output of several masters can be merged by another master. We exploit the properties of volumetric algorithms, i.e. an efficient exploitation of inter-frame redundancy and an efficient merging of partial models, to increase performances. We test our volumetric architecture with an innovative implementation of the visual hull. We use a label that codes simultaneously occupancy, subdivision of space and visibility allowing each camera to see only part of the volume of interest. We test our system on a particular implementation of the framework composed of eight cameras and twelve cores, two per acquisition node and four for the master. We achieve fifteen reconstructions per second and less than 100 ms latency from segmentation to the display of the reconstructed model. System performances have been measured on sequences of more than 20 000 frames with unconstrained user’s movements. The system computes a fair approximation of the user in all situations. / La structure tridimensionnelle du monde fait de la 3D l’évolution naturelle d’un grand nombre d’applications. De nombreux algorithmes de reconstruction différents ont été développés pour satisfaire les besoins de différents types d’applications. On cible les applications immersives: Les modèles reconstruits de l’utilisateur ainsi que des objets qu’il porte sont insérés dans des environnements virtuels créant ainsi la réalité mixte. Cette dernière est affichée ce qui permet à l’utilisateur d’interagir avec des éléments virtuels. Le caractère immersif de l’application requiert une reconstruction temps réel à faible latence. Qui plus est, on désire la reconstruction de modèles de formes arbitraires à l’aide de méthodes géométriques. L’état de l’art montre que les différentes caméras ne peuvent pas être connectées à un seul ordinateur responsable de la reconstruction 3D. L’application requiert un cluster d’ordinateurs et une méthode étudiée afin de répartir la charge de travail et d’optimiser les performances. J’ai développé au cours de ma thèse une architecture volumétrique distribuée pour reconstruire des objets de forme arbitraire en temps réel et avec faible latence. Le système exploite les propriétés des algorithmes volumétriques pour améliorer les performances. Il est composé de deux types de nœuds: acquisition et fusion. Les nœuds de type acquisition reconstruisent des modèles partiels et envoient des informations non redondantes aux nœuds de type fusion. Chaque nœud fusion est responsable de la création d’un modèle global à partir de plusieurs modèles partiels. Grâce aux algorithmes développés, chaque étape de fusion rassemble les informations de plusieurs modèles partiels tout en ayant un impact faible sur la latence totale du système. L’architecture a été testée avec une implémentation particulière du Visual Hull permettant une disposition plus libre des caméras. Ces dernières ne doivent plus observer l’entièreté du volume d’intérêt. Chaque partie de l’utilisateur est modélisée grâce aux caméras qui l’observent. Notre système dispose de quatre nœuds d’acquisitions dual core et d’un nœud de fusion dual CPU dual core. On reconstruit quinze modèles par seconde avec une latence inférieure à 100 ms mesurée depuis la segmentation jusqu’au rendu. Quelques soient les mouvements de l’utilisateur et les objets qu’il manipule, le système est capable de le modéliser.

In this thesis, we investigate the formulation, optimization and ambiguities in single-image 3D surface reconstruction from geometric and photometric constraints. We examine linear, discrete and quadratic constraints for shape from planar curves, shape from texture, and shape from shading. The problem of recovering 3D shape from the projection of planar curves on a surface is strongly motivated by perception studies. Applications include single-view modeling and uncalibrated structured light. When the curves intersect, the problem leads to a linear system for which a direct least-squares method is sensitive to noise. We derive a more stable solution and show examples where the same method produces plausible surfaces from the projection of parallel (non-intersecting) planar cross sections. The problem of reconstructing a smooth surface under constraints that have discrete ambiguities arise in areas such as shape from texture, shape from shading, photometric stereo and shape from defocus. While the problem is computationally hard, heuristics based on semidefinite programming may reveal the shape of the surface. Finally, we examine the shape from shading problem without boundary conditions as a polynomial system. This formulation allows, in generic cases, a complete solution for ideal polyhedral objects. For the general case we propose a semidefinite programming relaxation procedure, and an exact line search iterative procedure with a new smoothness term that favors folds at edges. We use this numerical technique to inspect shading ambiguities.

Techniques for fast, 3D, quantitative microscopy are of great interest in many fields. In this context, in-line digital holography has significant potential due to its relatively simple setup (lensless imaging), its three-dimensional character and its temporal resolution. The goal of this thesis is to improve existing hologram reconstruction techniques by employing an "inverse problems" approach. For applications of objects with parametric shapes, a greedy algorithm has been previously proposed which solves the (inherently ill-posed) inversion problem of reconstruction by maximizing the likelihood between a model of holographic patterns and the measured data. The first contribution of this thesis is to reduce the computational costs of this algorithm using a multi-resolution approach (FAST algorithm). For the second contribution, a "matching pursuit" type of pattern recognition approach is proposed for hologram reconstruction of volumes containing parametric objects, or non-parametric objects of a few shape classes. This method finds the closest set of diffraction patterns to the measured data using a diffraction pattern dictionary. The size of the dictionary is reduced by employing a truncated singular value decomposition to obtain a low cost algorithm. The third contribution of this thesis was carried out in collaboration with the laboratory of fluid mechanics and acoustics of Lyon (LMFA). The greedy algorithm is used in a real application: the reconstruction and tracking of free-falling, evaporating, ether droplets. In all the proposed methods, special attention has been paid to improvement of the accuracy of reconstruction as well as to reducing the computational costs and the number of parameters to be tuned by the user (so that the proposed algorithms are used with little or no supervision). A Matlab® toolbox (accessible on-line) has been developed as part of this thesis

Thèse numérisée par la Division de la gestion de documents et des archives de l'Université de Montréal.

L’objectif général de cette thèse est d’appliquer une méthode d’optimisation polynomiale basée sur la théorie des moments à certains problèmes de vision artificielle. Ces problèmes sont en général non convexes et classiquement résolus à l’aide de méthodes d’optimisation locales Ces techniques ne convergent généralement pas vers le minimum global et nécessitent de fournir une estimée initiale proche de la solution exacte. Les méthodes d’optimisation globale permettent d’éviter ces inconvénients. L’optimisation polynomiale basée sur la théorie des moments présente en outre l’avantage de prendre en compte des contraintes. Dans cette thèse nous étendrons cette méthode aux problèmes de minimisation d’une somme d’un grand nombre de fractions rationnelles. De plus, sous certaines hypothèses de "faible couplage" ou de "parcimonie" des variables du problème, nous montrerons qu’il est possible de considérer un nombre important de variables tout en conservant des temps de calcul raisonnables. Enfin nous appliquerons les méthodes proposées aux problèmes de vision par ordinateur suivants : minimisation des distorsions projectives induites par le processus de rectification d’images, estimation de la matrice fondamentale, reconstruction 3D multi-vues avec et sans distorsions radiales. / The overall objective of this thesis is to apply a polynomial optimization method, based on moments theory, on some vision problems. These problems are often nonconvex and they are classically solved using local optimization methods. Without additional hypothesis, these techniques don’t converge to the global minimum and need to provide an initial estimate close to the exact solution. Global optimization methods overcome this drawback. Moreover, the polynomial optimization based on moments theory could take into account particular constraints. In this thesis, we extend this method to the problems of minimizing a sum of many rational functions. In addition, under particular assumptions of "sparsity", we show that it is possible to deal with a large number of variables while maintaining reasonable computation times. Finally, we apply these methods to particular computer vision problems: minimization of projective distortions due to image rectification process, Fundamental matrix estimation, and multi-view 3D reconstruction with and without radial distortions.

Une des applications de l’échographie médicale est celle de l’ophtalmologie qui pose de nombreux problèmes spécifiques liés en partie à la faible dimension de l’oeil et à la précision importante que requièrent les mesures intraoculaires. En effet, avec le développement de la chirurgie réfractive qui regroupe ensemble des techniques capables de corriger les erreurs de réfraction et l’avènement des implants intraoculaires, le chirurgien ophtalmologiste est amené à surveiller la tolérance et les effets secondaires de ces implants sur les structures du segment antérieur. L’échographie à haute fréquence apporte la résolution suffisante pour cette tâche. Cependant, le développement de l’échographie 3D permet une extension des applications ophtalmologiques notamment pour le dimensionnement des implants en préopératoire. La modélisation 3D du segment antérieur permet d’étudier le comportement des implants et surtout de dessiner à terme un implant « sur mesure » pour le patient. C’est dans ce contexte que nous présentons une méthode originale de segmentation et de reconstruction 3D du segment antérieur par échographique haute fréquence en utilisant l’ajustement de modèles 3D. Nous utilisons un système échographique 3D de type main-libre, composé d’une sonde échographique haute fréquence, et d’un module de localisation actif comprenant une caméra et des marqueurs infrarouges. Ce système échographique 3D nous permet d’obtenir des images avec des informations de positionnement dans l’espace tridimensionnel associées. Nous avons ainsi pu mettre en place toute une chaîne d’acquisitions et de traitements des images échographiques. Nous créons, à partir d’images échographiques du segment antérieur oculaire, des modèles de référence 3D réalistes. Nous proposons ainsi une méthode d’ajustement de modèles 3D de référence sur des données 3D échographiques via l’utilisation de l’algorithme de recalage ICP. Nous avons également sélectionné et adapté différentes méthodes pour l’évaluation de l’approche de reconstruction proposée. Ces méthodes permettent de mettre en valeur la précision de ces reconstructions. / Ophthalmology is one of the clinical application fields of ultrasound imaging, for which numerous specific issues arise, related in part to the eye’s small anatomical dimensions combined with the high level of accuracy requirements associated with intraocular measurements. Indeed, since the development of refractive surgery including all the techniques dedicated to the correction of refractive errors, as well as the emergence of intraocular lens (IOL), ophthalmic surgeons have to monitor overall acceptance as well as secondary effects related to these implants on the structures of the anterior eye segment. High frequency ultrasound imaging provides the required spatial resolution for this task. However, the development of 3D ultrasound imaging allows for the development of new applications in ophthalmology, for instance pre-operative dimensioning of the lens. 3D modelling of the anterior eye segment therefore allows studying the IOL behaviour and may help designing future personalized IOL tailored for each patient. Within this context, we present an original 3D segmentation and reconstruction method based on 3D models registration, dedicated to the anterior eye segment acquired in high frequency ultrasound imaging. We used a 3D ultrasound free-hand acquisition system, composed of a high frequency ultrasound probe and a localization module based on a camera and infrared markers. This 3D ultrasound system provides images along with associated 3D spatial positioning information. We were therefore able to develop an entire ultrasound images acquisition and processing chain. This allowed us creating realistic reference 3D models from sequences of ultrasound images of the anterior eye segment. We thus propose a method based on the iterative closest point (ICP) algorithm for the registration of the 3D reference models to 3D ultrasound acquired data. We have also selected and adapted various methods for the evaluation of the proposed reconstruction process. These methods highlight the accuracy of the obtained reconstructions.

Dans le cadre de l'acquisition de l'information de profondeur de scènes texturées, un processus d'estimation de la profondeur basé sur la méthode de reconstruction 3D « Shape from Focus » est présenté dans ce manuscrit. Les deux étapes fondamentales de cette approche sont l'acquisition de la séquence d'images de la scène par sectionnement optique et l'évaluation de la netteté locale pour chaque pixel des images acquises. Deux systèmes d'acquisition de cette séquence d'images sont présentés ainsi que les traitements permettant d'exploiter celle-ci pour la suite du processus d'estimation de la profondeur. L'étape d'évaluation de la netteté des pixels passe par la comparaison des différents opérateurs de mesure de netteté. En plus des opérateurs usuels, deux nouveaux opérateurs basés sur les descripteurs généralisés de Fourier sont proposés. Une méthode nouvelle et originale de comparaison est développée et permet une analyse approfondie de la robustesse à différents paramètres des divers opérateurs. Afin de proposer une automatisation du processus de reconstruction, deux méthodes d'évaluation automatique de la netteté sont détaillées. Finalement, le processus complet de reconstruction est appliqué à des scènes agronomiques, mais également à une problématique du domaine de l'analyse de défaillances de circuits intégrés afin d'élargir les domaines d'utilisation / In the context of the acquisition of depth information for textured scenes, a depth estimation process based on a 3D reconstruction method called "shape from focus" is proposed in this thesis. The two crucial steps of this approach are the image sequence acquisition of the scene by optical sectioning and the local sharpness evaluation for each pixel of the acquired images. Two acquisition systems have been developed and are presented as well as different image processing techniques that enable the image exploitation for the depth estimation process. The pixel sharpness evaluation requires comparison of different focus measure operators in order to determine the most appropriate ones. In addition to the usual focus measure operators, two news operators based on generalized Fourier descriptors are presented. A new and original comparison method is developped and provides a further analysis of the robustness to various parameters of the focus measure operators. In order to provide an automatic version of the reconstruction process, two automatic sharpness evaluation methods are detailed. Finally, the whole reconstruction process is applied to agronomic scenes, but also to a problematic in failure analysis domain aiming to expand to other applications

Le phénomène d'îlot de chaleur urbain engendre de l'inconfort thermique auprès des habitants. Améliorer le microclimat en zone urbaine est donc l'une des préoccupations des aménageurs. La végétalisation des villes s'avère une solution prometteuse, car l'évapotranspiration des plantes etles ombres portées des arbres ont un impact significatif sur le bilan thermique de l'atmosphère alentour. Un défi majeur aujourd'hui est le développement d'un modèle de simulation microclimatique capable de reproduire les conditions climatiques d'une rue, voire d'un quartier urbain végétalisé, dans l'objectif de proposer un outil d'aide à la décision pour l'aménagement des villes durables. L'objectif de cette thèse est d'étudier comment prendre en compte la végétation et plus particulièrement les arbres, dans un modèle microclimatique 30 afin de simuler le microclimat d'un quartier. Deux modèles, LASER/F et RATP sont appliqués à l'échelle d'un arbre et d'un parc urbain pour mener à bien cette étude. / The urban heat island phenomenon causes thermal discomfort to residents. lmproving the microclimate in urban areas is therefore one of the concerns of urban plan ners. The greening of cities (with lawns, trees, green roofs, etc.) is a promising solution, as the transpiration of plants and the shadows of trees have a significant impact on the thermal balance of the surrounding atmosphere. This act must be planned to optimize the benefits of vegetation. A key challenge today is thus the development of a microclimatic simulation model capable of reproducing the climatic conditions of a street, or even a vegetated urban neighborhood, with the aim of proposinga decision support tool for the development of sustainable cities. The aim of this thesis is to study how to consider vegetation and especially trees, in a 30 microclimatic model to simulate the microclimate of a neighborhood. Two models, LASER/F and RATP were applied at the scale of a tree and an urban park to carry out this study.