Omni-directional (O-D) infrared (IR) vision is an effective capability for mobile systems in robotics, due to its advantages: illumination invariance, wide field-of-view, ease of identifying heat-emitting objects, and long term tracking without interruption. Unfortunately, O-D IR sensors have low resolution, low frame rates, high cost, sensor noise, and an increase in tracking time. In order to overcome these disadvantages, we propose an autonomous system application in indoor scenarios including 1) Dynamic 3D Reconstruction (D3DR) of the target view in real time images, 2) Human Behavior-based Target Tracking from O-D thermal images, 3) Thermal Multisensor Fusion (TMF), and 4) Visual Perception for Social Cognition from the motion behavior of the human target.

This thesis presents an approach for configuring and calibrating a network of RGB-D sensors used to guide a robotic arm to interact with objects that get rapidly modeled in 3D. The system is based on Microsoft Kinect sensors for 3D data acquisition. The work presented here also details an analysis and experimental study of the Kinect’s depth sensor capabilities and performance. The study comprises examination of the resolution, quantization error, and random distribution of depth data. In addition, the effects of color and reflectance characteristics of an object are also analyzed. The study examines two versions of Kinect sensors, one dedicated to operate with the Xbox 360 video game console and the more recent Microsoft Kinect for Windows version. The study of the Kinect sensor is extended to the design of a rapid acquisition system dedicated to large workspaces by the linkage of multiple Kinect units to collect 3D data over a large object, such as an automotive vehicle. A customized calibration method for this large workspace is proposed which takes advantage of the rapid 3D measurement technology embedded in the Kinect sensor and provides registration accuracy between local sections of point clouds that is within the range of the depth measurements accuracy permitted by the Kinect technology. The method is developed to calibrate all Kinect units with respect to a reference Kinect. The internal calibration of the sensor in between the color and depth measurements is also performed to optimize the alignment between the modalities. The calibration of the 3D vision system is also extended to formally estimate its configuration with respect to the base of a manipulator robot, therefore allowing for seamless integration between the proposed vision platform and the kinematic control of the robot. The resulting vision-robotic system defines the comprehensive calibration of reference Kinect with the robot. The latter can then be used to interact under visual guidance with large objects, such as vehicles, that are positioned within a significantly enlarged field of view created by the network of RGB-D sensors. The proposed design and calibration method is validated in a real world scenario where five Kinect sensors operate collaboratively to rapidly and accurately reconstruct a 180 degrees coverage of the surface shape of various types of vehicles from a set of individual acquisitions performed in a semi-controlled environment, that is an underground parking garage. The vehicle geometrical properties generated from the acquired 3D data are compared with the original dimensions of the vehicle.

La thèse propose une étude de l’architecture navale égyptienne de la Basse Époque (722-332 avant J.C.). Les vestiges iconographiques, épigraphiques et archéologiques existant sur ce sujet sont très maigres. Ainsi les recherches proposées se fondent principalement sur des nouvelles données archéologiques. Les fouilles menées par l’Institut d’Archéologie Sous-Marine (IEASM) en Égypte ont permis de découvrir plus de soixante bateaux datés du XIe au Ier s. av. J.-C dans les ports et les canaux de la ville engloutie de Thônis-Héracléion. En excellent état de conservation, il s’agit de la plus grande accumulation des bateaux antiques jamais retrouvés et au moins dix-huit d’entre eux sont datables de la Basse Epoque. La construction du bateau 17, qui constitue le sujet principal de la thèse, a été étudiée pendant trois saisons des fouilles (2009-2011). Le volume des nouvelles informations acquises est considérable et permet non seulement d’analyser en détail la construction du bateau en question mais aussi de tirer les conclusions plus générales concernant les principes et les méthodes de l’architecture navale en Egypte caractéristiques de cette époque. Plusieurs décisions techniques restent uniques à la tradition d’architecture navale égyptienne ainsi que l’est le choix d’essences locales pour la construction. Les traits de construction analysés dans la thèse sont systématiquement comparés aux documents iconographiques, épigraphiques et archéologiques disponibles. Il faut souligner l’importance de la description du bateau égyptien « baris » par Hérodote (« Histoires » II.96, env. 450 av. J.-C.) qui correspond bien à la construction du bateau 17 d’Héracleion. La première version du modèle 3D de ce bateau permet d’évaluer ses capacités techniques principales. La thèse proposera une comparaison entre le génie maritime égyptien de la Basse Epoque et ceux de la Méditerranée contemporaine. / The thesis is devoted to the study of Ancient Egyptian boatbuilding during the Late Period (722-332 BC). Iconographic, epigraphic and archaeological evidence on the subject being so scarce, the thesis is based mainly on a recently acquired archaeological data. Underwater excavations by the European Institute of Underwater Archaeology (IEASM) in Egypt allowed locating more than sixty ships dating from the 11th to 1st century BC in the harbour area and channels of the submerged city of Thonis-Heracleion. At least eighteen of these well-preserved ships are dated to the Late Period. The construction of ship 17 that has been studied during three excavation seasons (2009-2011) constitutes the main subject of current thesis. The considerable volume of new data allows not only to consider in detail the construction of the ship in question but also to propose several general conclusions on the constructional principles and methods characteristic to the Late Period. Many of these are unique to the Ancient Egyptian boatbuilding tradition as is the choice of local wood species for the construction. New archaeological data is compared throughout the research to existing iconographic, epigraphic and archaeological evidence. It is necessary to underline the importance of the description by Herodotus (“Histories” II.96, c.450 BC) of the local Egyptian ship “baris” as it corresponds well to the construction of ship 17. First version of the three-dimensional model of this ship allows estimating its major technical characteristics. The thesis proposes a comparison between the Late Egyptian and contemporary shipbuilding traditions of the Mediterranean.

Le développement de techniques de microscopie quantitatives tridimensionnelles et résolues en temps est fondamental dans de nombreux domaines. Dans ce cadre, l’holographie numérique en ligne recèle un fort potentiel, en raison de sa relative simplicité de mise en œuvre (imagerie sans lentille), de son caractère tridimensionnel et de sa résolution temporelle. Le but de cette thèse est l’amélioration des algorithmes de reconstruction des hologrammes par une approche « problèmes inverses ». Dans le cadre de la reconstruction d’objets paramétriques, des travaux antérieurs ont permis de proposer un algorithme glouton permettant de résoudre le problème inverse de reconstruction (intrinsèquement mal posé) par une maximisation de la vraisemblance entre un modèle de formation d’hologramme et les données. Une première contribution de ce travail de thèse a été de réduire le temps de calcul de cet algorithme en utilisant une approche multi-résolution (algorithme FAST). Dans une deuxième contribution, une approche reconnaissance de forme de type « matching pursuit » est utilisée pour la reconstruction d’objets quelconques en recherchant les éléments d’un dictionnaire les plus proches des figures de diffraction composant l’hologramme. La réduction des dimensions du dictionnaire est proposée en utilisant une décomposition en valeurs singulières tronquée. La troisième contribution de cette thèse a été réalisée en collaboration avec le LMFA. L’algorithme glouton a été utilisé sur un cas réel : la reconstruction et le suivi de gouttelettes d’éther évaporantes en chute libre. Dans tous ces développements une attention particulière a été portée sur la précision des reconstructions, sur la réduction du nombre de paramètres à régler par l’utilisateur (algorithmes peu ou non supervisés). Une boîte à outils Matlab® (en ligne) a été développée dans le cadre de cette thèse / Techniques for fast, 3D, quantitative microscopy are of great interest in many fields. In this context, in-line digital holography has significant potential due to its relatively simple setup (lensless imaging), its three-dimensional character and its temporal resolution. The goal of this thesis is to improve existing hologram reconstruction techniques by employing an “inverse problems” approach. For applications of objects with parametric shapes, a greedy algorithm has been previously proposed which solves the (inherently ill-posed) inversion problem of reconstruction by maximizing the likelihood between a model of holographic patterns and the measured data. The first contribution of this thesis is to reduce the computational costs of this algorithm using a multi-resolution approach (FAST algorithm). For the second contribution, a “matching pursuit” type of pattern recognition approach is proposed for hologram reconstruction of volumes containing parametric objects, or non-parametric objects of a few shape classes. This method finds the closest set of diffraction patterns to the measured data using a diffraction pattern dictionary. The size of the dictionary is reduced by employing a truncated singular value decomposition to obtain a low cost algorithm. The third contribution of this thesis was carried out in collaboration with the laboratory of fluid mechanics and acoustics of Lyon (LMFA). The greedy algorithm is used in a real application: the reconstruction and tracking of free-falling, evaporating, ether droplets. In all the proposed methods, special attention has been paid to improvement of the accuracy of reconstruction as well as to reducing the computational costs and the number of parameters to be tuned by the user (so that the proposed algorithms are used with little or no supervision). A Matlab® toolbox (accessible on-line) has been developed as part of this thesis

Le relevé et la reconstruction 3D de scènes sous-marine deviennent chaque jour plus incontournable devant notre intérêt grandissant pour l’étude des fonds sous-marins. La majorité des travaux existants dans ce domaine sont fondés sur l’utilisation de capteurs acoustiques l’image n’étant souvent qu’illustrative.L’objectif de cette thèse consiste à développer des techniques permettant la fusion de données hétérogènes issues d’un système photogrammétrique et d’un système acoustique.Les travaux présentés dans ce mémoire sont organisés en trois parties. La première est consacrée au traitement des données 2D afin d’améliorer les couleurs des images sous-marines pour augmenter la répétabilité des descripteurs en chaque point 2D. Puis, nous proposons un système de visualisation de scène en 2D sous forme de mosaïque.Dans la deuxième partie, une méthode de reconstruction 3D à partir d’un ensemble non ordonné de plusieurs images a été proposée. Les données 3D ainsi calculées seront fusionnées avec les données provenant du système acoustique dans le but de reconstituer le site sous-marin.Dans la dernière partie de ce travail de thèse, nous proposons une méthode de recalage 3D originale qui se distingue par la nature du descripteur extrait en chaque point. Le descripteur que nous proposons est invariant aux transformations isométriques (rotation, transformation) et permet de s’affranchir du problème de la multi-résolution. Nous validons à l’aide d’une étude effectuée sur des données synthétiques et réelles où nous montrons les limites des méthodes de recalages existantes dans la littérature. Au final, nous proposons une application de notre méthode à la reconnaissance d’objets 3D. / The survey and the 3D reconstruction of underwater become indispensable for our growing interest in the study of the seabed. Most of the existing works in this area are based on the use of acoustic sensors image.The objective of this thesis is to develop techniques for the fusion of heterogeneous data from a photogrammetric system and an acoustic system.The presented work is organized in three parts. The first is devoted to the processing of 2D data to improve the colors of the underwater images, in order to increase the repeatability of the feature descriptors. Then, we propose a system for creating mosaics, in order to visualize the scene.In the second part, a 3D reconstruction method from an unordered set of several images was proposed. The calculated 3D data will be merged with data from the acoustic system in order to reconstruct the underwater scene.In the last part of this thesis, we propose an original method of 3D registration in terms of the nature of the descriptor extracted at each point. The descriptor that we propose is invariant to isometric transformations (rotation, transformation) and addresses the problem of multi-resolution. We validate our approach with a study on synthetic and real data, where we show the limits of the existing methods of registration in the literature. Finally, we propose an application of our method to the recognition of 3D objects.

Este documento apresenta uma dissertação sobre o desenvolvimento de um sistema de integração de dados para geração de estruturas tridimensionais a partir de vídeo 3D. O trabalho envolve a extensão de um sistema de vídeo 3D em tempo real proposto recentemente. Esse sistema, constituído por projetor e câmera, obtém imagens de profundidade de objetos por meio da projeção de slides com um padrão de faixas coloridas. Tal procedimento permite a obtenção, em tempo real, tanto do modelo 2,5 D dos objetos quanto da textura dos mesmos, segundo uma técnica denominada luz estruturada. Os dados são capturados a uma taxa de 30 quadros por segundo e possuem alta qualidade: resoluções de 640 x 480 pixeis para a textura e de 90 x 240 pontos (em média) para a geometria. A extensão que essa dissertação propõe visa obter o modelo tridimensional dos objetos presentes em uma cena por meio do registro dos dados (textura e geometria) dos diversos quadros amostrados. Assim, o presente trabalho é um passo intermediário de um projeto maior, no qual pretende-se fazer a reconstrução dos modelos por completo, bastando para isso apenas algumas imagens obtidas a partir de diferentes pontos de observação. Tal reconstrução deverá diminuir a incidência de pontos de oclusão (bastante comuns nos resultados originais) de modo a permitir a adaptação de todo o sistema para objetos móveis e deformáveis, uma vez que, no estado atual, o sistema é robusto apenas para objetos estáticos e rígidos. Até onde pudemos averiguar, nenhuma técnica já foi aplicada com este propósito. Este texto descreve o trabalho já desenvolvido, o qual consiste em um método para detecção, rastreamento e casamento espacial de componentes conexas presentes em um vídeo 3D. A informação de imagem do vídeo (textura) é combinada com posições tridimensionais (geometria) a fim de alinhar partes de superfícies que são vistas em quadros subseqüentes. Esta é uma questão chave no vídeo 3D, a qual pode ser explorada em diversas aplicações tais como compressão, integração geométrica e reconstrução de cenas, dentre outras. A abordagem que adotamos consiste na detecção de características salientes no espaço do mundo, provendo um alinhamento de geometria mais completo. O processo de registro é feito segundo a aplicação do algoritmo ICP---Iterative Closest Point---introduzido por Besl e McKay em 1992. Resultados experimentais bem sucedidos corroborando nosso método são apresentados. / This document presents a MSc thesis focused on the development of a data integration system to generate tridimensional structures from 3D video. The work involves the extension of a recently proposed real time 3D video system. This system, composed by a video camera and a projector, obtains range images of recorded objects using slide projection of a coloured stripe pattern. This procedure allows capturing, in real time, objects´ texture and 2,5 D model, at the same time, by a technique called structured light. The data are acquired at 30 frames per second, being of high quality: the resolutions are 640 x 480 pixels and 90 x 240 points (in average), respectively. The extension that this thesis proposes aims at obtaining the tridimensional model of the objects present in a scene through data matching (texture and geometry) of various sampled frames. Thus, the current work is an intermediary step of a larger project with the intent of achieving a complete reconstruction from only a few images obtained from different viewpoints. Such reconstruction will reduce the incidence of occlusion points (very common on the original results) such that it should be possible to adapt the whole system to moving and deformable objects (In the current state, the system is robust only to static and rigid objects.). To the best of our knowledge, there is no method that has fully solved this problem. This text describes the developed work, which consists of a method to perform detection, tracking and spatial matching of connected components present in a 3D video. The video image information (texture) is combined with tridimensional sites (geometry) in order to align surface portions seen on subsequent frames. This is a key step in the 3D video that may be explored in several applications such as compression, geometric integration and scene reconstruction, to name but a few. Our approach consists of detecting salient features in both image and world spaces, for further alignment of texture and geometry. The matching process is accomplished by the application of the ICP---Iterative Closest Point---algorithm, introduced by Besl and McKay in 1992. Succesful experimental results corroborating our method are shown.

This thesis deals with the task of three-dimensional scene reconstruction using image data obtained from multiple views. It is assumed that intrinsic parameters of the utilized cameras are known. The theoretical chapters describe the basic priciples of individual reconstruction steps. Variuous possible implementaions of data model suitable for this task are also described. The practical part also includes a comparison of false keypoint correspondence filtering, implementation of polar stereo rectification and comparison of disparity map calculation methods that are bundled with the OpenCV library. In the final portion of the thesis, examples of reconstructed 3D models are presented and discussed.

Cette thèse s'intéresse à deux problèmes rencontrés en traitement du signal et des images. Le premierconcerne la reconstruction 3D de structures biologiques à partir d'acquisitions multi-angles enmicroscopie par réflexion totale interne (MA-TIRF). Dans ce contexte, nous proposons de résoudre leproblème inverse avec une approche variationnelle et étudions l'effet de la régularisation. Une batteried'expériences, simples à mettre en oeuvre, sont ensuite proposées pour étalonner le système et valider lemodèle utilisé. La méthode proposée s'est montrée être en mesure de reconstruire avec précision unéchantillon phantom de géométrie connue sur une épaisseur de 400 nm, de co-localiser deux moléculesfluorescentes marquant les mêmes structures biologiques et d'observer des phénomènes biologiquesconnus, le tout avec une résolution axiale de l'ordre de 20 nm. La deuxième partie de cette thèseconsidère plus précisément la régularisation l0 et la minimisation du critère moindres carrés pénalisé (l2-l0) dans le contexte des relaxations continues exactes de cette fonctionnelle. Nous proposons dans unpremier temps la pénalité CEL0 (Continuous Exact l0) résultant en une relaxation de la fonctionnelle l2-l0 préservant ses minimiseurs globaux et pour laquelle de tout minimiseur local on peut définir unminimiseur local de l2-l0 par un simple seuillage. Par ailleurs, nous montrons que cette relaxation éliminedes minimiseurs locaux de la fonctionnelle initiale. La minimisation de cette fonctionnelle avec desalgorithmes d'optimisation non-convexe est ensuite utilisée pour différentes applications montrantl'intérêt de la minimisation de la relaxation par rapport à une minimisation directe du critère l2-l0. Enfin,une vue unifiée des pénalités continues de la littérature est proposée dans ce contexte de reformulationexacte du problème / This thesis is devoted to two problems encountered in signal and image processing. The first oneconcerns the 3D reconstruction of biological structures from multi-angle total interval reflectionfluorescence microscopy (MA-TIRF). Within this context, we propose to tackle the inverse problem byusing a variational approach and we analyze the effect of the regularization. A set of simple experimentsis then proposed to both calibrate the system and validate the used model. The proposed method hasbeen shown to be able to reconstruct precisely a phantom sample of known geometry on a 400 nmdepth layer, to co-localize two fluorescent molecules used to mark the same biological structures andalso to observe known biological phenomena, everything with an axial resolution of 20 nm. The secondpart of this thesis considers more precisely the l0 regularization and the minimization of the penalizedleast squares criteria (l2-l0) within the context of exact continuous relaxations of this functional. Firstly,we propose the Continuous Exact l0 (CEL0) penalty leading to a relaxation of the l2-l0 functional whichpreserves its global minimizers and for which from each local minimizer we can define a local minimizerof l2-l0 by a simple thresholding. Moreover, we show that this relaxed functional eliminates some localminimizers of the initial functional. The minimization of this functional with nonsmooth nonconvexalgorithms is then used on various applications showing the interest of minimizing the relaxation incontrast to a direct minimization of the l2-l0 criteria. Finally we propose a unified view of continuouspenalties of the literature within this exact problem reformulation framework

During cell division, chromosome segregation takes place on bipolar, microtubulebased spindles. Here, C. elegans is used to analyze spindle organization under both mitotic and meiotic conditions. First, the role of SAS-4 in organizing centrosome structure was analyzed. Partial depletion of SAS-4 in early embryos results in structurally defective centrioles. The study of this protein sheds light on the poorly understood role of the centrioles in dictating centrosome size. Second, the ultrastructure of wild-type mitotic spindle components was analyzed by electron tomography. This 3-D analysis reveals morphologically distinct microtubule end morphologies in the mitotic spindle pole. These results have structural implications for models of microtubule interactions with centrosomes Third, spindle assembly was studied in female meiosis. Specifically, the role of the microtubule severing complex katanin in spindle organization was analyzed. Electron tomography reveals fragmentation of spindle microtubules and suggests a novel katanin-dependent mechanism of meiotic spindle assembly. In this model, relatively long microtubules seen near the meiotic chromatin are converted into numerous short fragments, thus increasing the total number of polymers in an acentrosomal environment. Taken together, these results provide novel insights into the three-dimensional organization of microtubules during spindle assembly. / Die Segregation der Chromosomen während der Zellteilung wird duch bipolare, von Microtubuli-aufgebauten Spindlen gewährleistet. In der vorliegenden Arbeit wird C. elegans zur Analyse der Spindelorganisation unter mitotischen und meiotischen Bedingungen herangezogen. Erstens wird die Rolle von SAS-4 in der Organisation von Zentrosomen untersucht. Die partielle Depletierung von SAS-4 in frühen Embryonen führt zu strukturell defekten Zentriolen und wirft somit Licht auf die wenig verstandene Rolle der Zentriolen in der Bestimmung der Zentrosomengröße. Zweitens wird die Ultrastruktur der mitotischen Spindelkomponenten im Wildtyp durch Elektronentomographie untersucht. Diese 3-D-Analyse zeigt, dass im mitotischen Spindlepol unterschiedliche Morphologien der Mikrotubulienden zu finden sind. Diese Ergebnisse haben strukturelle Implikationen für Modelle der Mikrotubuli-Zentrosomen-Interaktionen. Drittens wird der Aufbau der Spindel in der weiblichen Meiose, speziell die Rolle des Mikrotubuli-schneidenden Kataninkomplexes in der Spindelorganisation, untersucht. Die Elektronentomographie zeigt hier eine Fragmentierung der Spindelmikrotubuli. Basierend auf diesem Ergebnis wird ein neues Katanin-abhängiges Modell der Formierung der Meiosespindel entwickelt, in dem relativ lange Microtubuli in Nähe des meiotischen Chromatins in zahlreiche kurze Mikrotubuli “zerschnitten” werden. Dies erhöht die Anzahl der verfügbaren Polymere in dieser azentrosomalen Situation. Zusammenfassend bringen diese Ergebnisse neue Einsichten in die räumliche Organisation der Mikrotubuli während des Spindelaufbaus.

No description available.