This thesis presents an approach for configuring and calibrating a network of RGB-D sensors used to guide a robotic arm to interact with objects that get rapidly modeled in 3D. The system is based on Microsoft Kinect sensors for 3D data acquisition. The work presented here also details an analysis and experimental study of the Kinect’s depth sensor capabilities and performance. The study comprises examination of the resolution, quantization error, and random distribution of depth data. In addition, the effects of color and reflectance characteristics of an object are also analyzed. The study examines two versions of Kinect sensors, one dedicated to operate with the Xbox 360 video game console and the more recent Microsoft Kinect for Windows version. The study of the Kinect sensor is extended to the design of a rapid acquisition system dedicated to large workspaces by the linkage of multiple Kinect units to collect 3D data over a large object, such as an automotive vehicle. A customized calibration method for this large workspace is proposed which takes advantage of the rapid 3D measurement technology embedded in the Kinect sensor and provides registration accuracy between local sections of point clouds that is within the range of the depth measurements accuracy permitted by the Kinect technology. The method is developed to calibrate all Kinect units with respect to a reference Kinect. The internal calibration of the sensor in between the color and depth measurements is also performed to optimize the alignment between the modalities. The calibration of the 3D vision system is also extended to formally estimate its configuration with respect to the base of a manipulator robot, therefore allowing for seamless integration between the proposed vision platform and the kinematic control of the robot. The resulting vision-robotic system defines the comprehensive calibration of reference Kinect with the robot. The latter can then be used to interact under visual guidance with large objects, such as vehicles, that are positioned within a significantly enlarged field of view created by the network of RGB-D sensors. The proposed design and calibration method is validated in a real world scenario where five Kinect sensors operate collaboratively to rapidly and accurately reconstruct a 180 degrees coverage of the surface shape of various types of vehicles from a set of individual acquisitions performed in a semi-controlled environment, that is an underground parking garage. The vehicle geometrical properties generated from the acquired 3D data are compared with the original dimensions of the vehicle.

Image segmentation is one of the most difficult tasks in image processing. Segmentation algorithms are generally based on searching a region where pixels share similar gray level intensity and satisfy a set of defined criteria. However, the segmented region cannot be used directly for partial image retrieval. In this dissertation, a Contour Based Image Structure (CBIS) model is introduced. In this model, images are divided into several objects defined by their bounding contours. The bounding contour structure allows individual object extraction, and partial object matching and retrieval from a standard CBIS image structure. The CBIS model allows the representation of 3D objects by their bounding contours which is suitable for parallel implementation particularly when extracting contour features and matching them for 3D images require heavy computations. This computational burden becomes worse for images with high resolution and large contour density. In this essence we designed two parallel algorithms; Contour Parallelization Algorithm (CPA) and Partial Retrieval Parallelization Algorithm (PRPA). Both algorithms have considerably improved the performance of CBIS for both contour shape matching as well as partial image retrieval. To improve the effectiveness of CBIS in segmenting images with inhomogeneous backgrounds we used the phase congruency invariant features of Fourier transform components to highlight boundaries of objects prior to extracting their contours. The contour matching process has also been improved by constructing a fuzzy contour matching system that allows unbiased matching decisions. Further improvements have been achieved through the use of a contour tailored Fourier descriptor to make translation and rotation invariance. It is proved to be suitable for general contour shape matching where translation, rotation, and scaling invariance are required. For those images which are hard to be classified by object contours such as bacterial images, we define a multi-level cosine transform to extract their texture features for image classification. The low frequency Discrete Cosine Transform coefficients and Zenike moments derived from images are trained by Support Vector Machine (SVM) to generate multiple classifiers.

Aquesta tesi tracta sobre la combinació del control visual i la llum estructurada. El control visual clàssic assumeix que elements visuals poden ser fàcilment extrets de les imatges. Això fa que objectes d'aspecte uniforme o poc texturats no es puguin tenir en compte. En aquesta tesi proposem l'ús de la llum estructurada per dotar d'elements visuals als objectes independentment de la seva aparença.En primer lloc, es presenta un ampli estudi de la llum estructurada, el qual ens permet proposar un nou patró codificat que millora els existents. La resta de la tesi es concentra en el posicionament d'un robot dotat d'una càmara respecte diferentsobjectes, utilitzant la informació proveïda per la projecció de diferents patrons de llum. Dos configuracions han estat estudiades: quan el projector de llum es troba separat del robot,i quan el projector està embarcat en el robot juntament amb la càmara. Les tècniques proposades en la tesi estan avalades per un ampli estudi analític i validades per resultats experimentals. / This thesis treats about the combination of visual servoing and structured light. Classic visual servoing assumes that visual features can be extracted from the images. However, uniform ornon-textured objects, or objects for which extracting features is too complex or too time consuming cannot be taken into account.This thesis proposes the use of structured light patterns for providing suitable visual features independently of the object appearance.Firstly, a comprehensive survey on coded structured light patterns is presented. Then, a new pattern improving the existing ones isproposed. The remaining of the thesis is devoted to position an eye-in-hand robot with respect to objects by using features provided by light patterns. Two configurations are tested. In thefirst one, an off-board video-projector is used while in the second, an onboard structured light emitter is exploited. The techniques proposed in the thesis are supported by theoreticalanalysis and they are validated by experimental results.

During cell division, chromosome segregation takes place on bipolar, microtubulebased spindles. Here, C. elegans is used to analyze spindle organization under both mitotic and meiotic conditions. First, the role of SAS-4 in organizing centrosome structure was analyzed. Partial depletion of SAS-4 in early embryos results in structurally defective centrioles. The study of this protein sheds light on the poorly understood role of the centrioles in dictating centrosome size. Second, the ultrastructure of wild-type mitotic spindle components was analyzed by electron tomography. This 3-D analysis reveals morphologically distinct microtubule end morphologies in the mitotic spindle pole. These results have structural implications for models of microtubule interactions with centrosomes Third, spindle assembly was studied in female meiosis. Specifically, the role of the microtubule severing complex katanin in spindle organization was analyzed. Electron tomography reveals fragmentation of spindle microtubules and suggests a novel katanin-dependent mechanism of meiotic spindle assembly. In this model, relatively long microtubules seen near the meiotic chromatin are converted into numerous short fragments, thus increasing the total number of polymers in an acentrosomal environment. Taken together, these results provide novel insights into the three-dimensional organization of microtubules during spindle assembly. / Die Segregation der Chromosomen während der Zellteilung wird duch bipolare, von Microtubuli-aufgebauten Spindlen gewährleistet. In der vorliegenden Arbeit wird C. elegans zur Analyse der Spindelorganisation unter mitotischen und meiotischen Bedingungen herangezogen. Erstens wird die Rolle von SAS-4 in der Organisation von Zentrosomen untersucht. Die partielle Depletierung von SAS-4 in frühen Embryonen führt zu strukturell defekten Zentriolen und wirft somit Licht auf die wenig verstandene Rolle der Zentriolen in der Bestimmung der Zentrosomengröße. Zweitens wird die Ultrastruktur der mitotischen Spindelkomponenten im Wildtyp durch Elektronentomographie untersucht. Diese 3-D-Analyse zeigt, dass im mitotischen Spindlepol unterschiedliche Morphologien der Mikrotubulienden zu finden sind. Diese Ergebnisse haben strukturelle Implikationen für Modelle der Mikrotubuli-Zentrosomen-Interaktionen. Drittens wird der Aufbau der Spindel in der weiblichen Meiose, speziell die Rolle des Mikrotubuli-schneidenden Kataninkomplexes in der Spindelorganisation, untersucht. Die Elektronentomographie zeigt hier eine Fragmentierung der Spindelmikrotubuli. Basierend auf diesem Ergebnis wird ein neues Katanin-abhängiges Modell der Formierung der Meiosespindel entwickelt, in dem relativ lange Microtubuli in Nähe des meiotischen Chromatins in zahlreiche kurze Mikrotubuli “zerschnitten” werden. Dies erhöht die Anzahl der verfügbaren Polymere in dieser azentrosomalen Situation. Zusammenfassend bringen diese Ergebnisse neue Einsichten in die räumliche Organisation der Mikrotubuli während des Spindelaufbaus.

The Lower Ordovician El Paso Group is a >400-m-thick carbonate succession exposed in the Franklin Mountains, El Paso, Texas. The El Paso Group contains multiple breccias related to collapsed-paleocave systems. These breccias have been documented as having formed during the top-Lower Ordovician Sauk depositional supersequence lowstand. Evidence presented in this study suggests that cave formation may have been as much as 350 million years younger and related to Laramide oblique right lateral compression. Regardless of the timing of formation, the breccias mapped in this study are of collapsed paleocave origin based on breccia clast organization and matrix content. Speleogenetic models are compared against observations of breccia distribution by direct field observations and mapping on sub-meter airborne light detection and ranging or lidar data. Point vectors were defined for every point within study area to highlight subtle changes in outcrop erosional profile for mapping geological features directly on the lidar point cloud. In addition, spectral data from airborne photography and hyperspectral image analysis were used assist in geological contact definition. A digital outcrop model was constructed from 3D geologic mapping results from which spatial statistic were extracted and used to reconstruct collapsed paleocave breccia bodies. The resultant breccia geometries were compared against laser-scanned modern cave dimensions, from Devil's Sinkhole, Rocksprings, Texas, and used in analysis of conceptual models for cave formation. The breccias of the southern Franklin Mountains follow linear trends that closely match Riedel shear fracture patterns predicted from right-lateral oblique compression. Stress orientations that match right-lateral oblique compression in the Phanerozoic of the El Paso region are related to the Laramide orogeny. The relationship of observed structures and the orientation of collapse breccias may indicate that southern Franklin Mountain breccia bodies are the result of a solution-enhanced tectonic karst system. / text

La scoliose idiopathique de l’adolescent est une déformation 3D du rachis. La littérature comporte une multitude d’études sur la prédiction de l’évolution et l’identification de facteurs de risque de progression. Pour l’instant les facteurs de risque établis sont l’amplitude de la déformation, la maturité squelettique et le type de courbure. Plusieurs autres champs ont été explorés comme les aspects génétiques, biochimiques, mécaniques, posturaux et topographiques, sans vraiment apporter beaucoup de précision à la prédiction de l’évolution. L’avancement de la technologie permet maintenant de générer des reconstructions 3D du rachis à l’aide des radiographies standard et d’obtenir des mesures de paramètres 3D. L’intégration de ces paramètres 3D dans un modèle prédictif représente une avenue encore inexplorée qui est tout à fait logique dans le contexte de cette déformation 3D du rachis. L’objectif général de cette thèse est de développer un modèle de prédiction de l’angle de Cobb à maturité squelettique à partir de l’information disponible au moment de la première visite, soit l’angle de Cobb initial, le type de courbure, l’âge osseux et des paramètres 3D du rachis. Dans une première étude, un indice d’âge osseux a été développé basé sur l’ossification de l’apophyse iliaque et sur le statut du cartilage triradié. Cet indice comporte 3 stades et le second stade, qui est défini par un cartilage triradié fermé avec maximum 1/3 d’ossification de l’apophyse iliaque, représente le moment pendant lequel la progression de la scoliose idiopathique de l’adolescent est la plus rapide. Une seconde étude rétrospective a permis de mettre en évidence le potentiel des paramètres 3D pour améliorer la prédiction de l’évolution. Il a été démontré qu’à la première visite il existe des différences pour 5 paramètres 3D du rachis entre un groupe de patients qui sera éventuellement opéré et un groupe qui ne progressera pas. Ces paramètres sont : la moyenne da la cunéiformisation 3D des disques apicaux, la rotation intervertébrale à la jonction inférieure de la courbure, la torsion, le ratio hauteur/largeur du corps vertébral de T6 et de la colonne complète. Les deux dernières études sont basées sur une cohorte prospective de 133 patients avec une scoliose idiopathique de l’adolescent suivi dès leur première visite à l’hôpital jusqu’à maturité squelettique. Une première étude a permis de mettre en évidence les différences morphologiques à la première visite entre les patients ayant progresser de plus ou moins de 6°. Des différences ont été mise en évidence pour la cyphose, l’angle de plan de déformation maximal, la rotation ntervertébrale l’apex, la torsion et plusieurs paramètres de «slenderness». Ensuite une seconde étude a permis de développer un modèle prédictif basé sur un modèle linéaire général en incluant l’indice d’âge osseux développé dans la première étude, le type de courbure, l’amplitude de l’angle de Cobb à la première visite, l’angle de déformation du plan maximale, la cunéiformisation 3D des disques T3-T4, T8-­T9, T11-­T12 et la somme des cunéiformisation 3D de tous les disques thoraciques et lombaires. Le coefficient de détermination multiple pour cette modélisation est de 0.715. Le modèle prédictif développé renforce l’importance de considérer la scoliose idiopathique dans les trois dimensions et il permettra d’optimiser la prédiction de l’évolution au moment de la première visite. / Prediction of curve progression remains a challenge for the clinicians at the first visit for a patient with adolescent idiopathic scoliosis. Prediction of progression is based on curve type, curve magnitude and skeletal or chronological age. The failure to accurately predict the risk of progression can lead to inadequate treatment, as well as unnecessary medical visits and radiographs. Three-­dimensional evaluation is currently more popular in the scoliosis research community either for classification or treatment planning. The global objective of this thesis was to develop a predictive model of the final Cobb angle in adolescent idiopathic scoliosis based on 3D spine parameters and on skeletal age, curve type and curve magnitude. The first study was to develop a skeletal maturity system based on ossification of iliac apophysis and triradiate cartilage status, two details available on routine radiographs. A 3 stages system was develop with the second stage being associated to the rapid acceleration phase of the deformation. The second stage corresponds to a closed triradiate cartilage and a maximum of 1/3 of the iliac apophysis ossification. The second study was a retrospective case control study. It showed the potential for 3D parameters to help predict the evolution. 3D spinal parameters at first visit of a group of patient eventually treated with surgery and a group who had no progression revealed statistical differences for: mean 3D wedging of the apical disks, lower junctional vertebral axial rotation, torsion and T6 and whole spine height/width ratio were all significantly affected. The last study was a prospective cohort study based on an adolescent idiopathic scoliosis group followed from first visit to skeletal maturity. A general linear model was developed based on 3D spinal parameters and on curve type, Cobb angle at first visit and with the 3 stages skeletal maturity system developed in the first study. The predictive model obtained has a determination coefficient of 0,715. Included 3D parameters predictors were: the angle of the plane of maximal curvature, the 3D wedging of T3-­T4, T8-­T9 and T11-­T12 disks, and the sum of 3D wedging of all thoracic and lumbar disks. The predictive model developed reinforced the importance of considering adolescent idiopathic scoliosis in a three-­‐dimensional point of view. This model will also optimise the prediction of evolution at first visit.

This thesis presents an approach for configuring and calibrating a network of RGB-D sensors used to guide a robotic arm to interact with objects that get rapidly modeled in 3D. The system is based on Microsoft Kinect sensors for 3D data acquisition. The work presented here also details an analysis and experimental study of the Kinect’s depth sensor capabilities and performance. The study comprises examination of the resolution, quantization error, and random distribution of depth data. In addition, the effects of color and reflectance characteristics of an object are also analyzed. The study examines two versions of Kinect sensors, one dedicated to operate with the Xbox 360 video game console and the more recent Microsoft Kinect for Windows version. The study of the Kinect sensor is extended to the design of a rapid acquisition system dedicated to large workspaces by the linkage of multiple Kinect units to collect 3D data over a large object, such as an automotive vehicle. A customized calibration method for this large workspace is proposed which takes advantage of the rapid 3D measurement technology embedded in the Kinect sensor and provides registration accuracy between local sections of point clouds that is within the range of the depth measurements accuracy permitted by the Kinect technology. The method is developed to calibrate all Kinect units with respect to a reference Kinect. The internal calibration of the sensor in between the color and depth measurements is also performed to optimize the alignment between the modalities. The calibration of the 3D vision system is also extended to formally estimate its configuration with respect to the base of a manipulator robot, therefore allowing for seamless integration between the proposed vision platform and the kinematic control of the robot. The resulting vision-robotic system defines the comprehensive calibration of reference Kinect with the robot. The latter can then be used to interact under visual guidance with large objects, such as vehicles, that are positioned within a significantly enlarged field of view created by the network of RGB-D sensors. The proposed design and calibration method is validated in a real world scenario where five Kinect sensors operate collaboratively to rapidly and accurately reconstruct a 180 degrees coverage of the surface shape of various types of vehicles from a set of individual acquisitions performed in a semi-controlled environment, that is an underground parking garage. The vehicle geometrical properties generated from the acquired 3D data are compared with the original dimensions of the vehicle.

L'objectiu d'aquesta tesi és l'estudi de les diferents tècniques per alinear vistes tridimensionals. Aquest estudi ens ha permès detectar els principals problemes de les tècniques existents, aprotant una solució novedosa i contribuint resolent algunes de les mancances detectades especialment en l'alineament de vistes a temps real. Per tal d'adquirir les esmentades vistes, s'ha dissenyat un sensor 3D manual que ens permet fer adquisicions tridimensionals amb total llibertat de moviments. Així mateix, s'han estudiat les tècniques de minimització global per tal de reduir els efectes de la propagació de l'error. / The goal of this thesis is to study the different techniques used to register 3D acquisitions. This study detects the main drawbacks of the existing techniques, presents a new classification and provides significant solutions of some perceived shortcomings, especially in 3D real time registration. A 3D hand-held sensor has been designed to acquire these views without any motion restriction and global minimization techniques have been studied to decrease the error propagation effects.

Universidade de Uberaba / In the past years, motion capture has been widely used in many application areas. In movies and games, motion capture is frequently employed to animate virtual characters. In sports, motion capture and analyses focus on optimizing movements of athletes and injury prevention. More applications areas include medicine, military and engineering. Motion capture can be accomplished by several technologies. However, optical marker-based systems are considered the gold standard of the motion capture field. They can offer high precision levels and flexibility to support most applications, but they are also the most expensive systems due to high costs of hardware and software. Although low-cost optical systems have been proposed in the last decade, these systems cannot provide enough precision, flexibility, automatism and/or real-time capability for a number of applications. In this context, the present research aims to develop a complete and high precision approach to track and reconstruct a cloud of independent markers, in real time, using multiple infrared specialized cameras. The proposed method includes a set of relatively simple algorithms which are part of a three-stage procedure. These stages work on the tracking and matching of the image points and spatial reconstruction of the marker trajectories. In order to evaluate the method, a prototype software has been implemented and experiments were performed using a pack of eight infrared cameras. The NaturalPoint s OptiTrack system, which includes the Arena software, was used as a reference system. In the experiments, the proposed technique was able to successfully track and reconstruct a set of 38 reflective markers in real time. When compared to the commercial software, the method performed better for automatically maker tracking. In addition, the reconstructed trajectories produced by the prototype software were far less contaminated by noise than the trajectories generated by the Arena software. The proposed method should encourage the development of new high performance systems at a more affordable price. / Nos últimos anos, os sistemas de captura de movimento vêm sendo aplicados em diversas áreas do conhecimento. A animação de personagens virtuais na indústria cinematográfica e a avaliação dos movimentos corporais em áreas da saúde e nos esportes são apenas algumas de suas diversas aplicações. Dos diferentes tipos de sistemas de captura de movimento existentes, os sistemas ópticos baseados em marcadores são reconhecidos como os mais avançados e os que oferecem níveis de precisão e flexibilidade suficientes para suportar o maior número de aplicações. Todavia, esses sistemas estão também entre os mais caros, devido ao alto preço dos elementos de hardware e software. As propostas de baixo custo existentes possuem limitações com relação à precisão, flexibilidade, automatismo e/ou velocidade de processamento, que as tornam inviáveis para uma considerável parcela de aplicações. Neste contexto, esta pesquisa propõe um método computacional completo, considerando as principais deficiências dos trabalhos existentes, para rastrear e reconstruir as trajetórias de uma nuvem de marcadores reflexivos independentes, em tempo real, utilizando múltiplas câmeras especializadas de infravermelho. A técnica proposta envolve um conjunto de algoritmos relativamente simples, os quais integram três etapas principais. Estas etapas realizam o rastreamento e o casamento dos pontos nas imagens e a reconstrução tridimensional das trajetórias dos marcadores. O método foi implementado em software e testado utilizando uma coleção de oito câmeras de infravermelho. Como referência, utilizou-se o software Arena e o sistema OptiTrack comercializado pela empresa NaturalPoint. Nos experimentos conduzidos, a técnica foi capaz de rastrear automaticamente um conjunto de 38 marcadores em tempo real. Comparada ao software comercial Arena, a técnica revelou uma melhor capacidade de rastreamento automático dos marcadores e ainda reconstruiu trajetórias com menor incidência de ruídos. O método proposto deve incentivar pesquisadores e empresas a desenvolverem sistemas de captura de movimento de alto desempenho e com menor custo. / Doutor em Ciências

This report shows a robust and efficient implementation of a speed-optimized algorithm for object recognition, 3D real world location and tracking in real time. It details a design that was focused on detecting and following objects in flight as applied to a football in motion. An overall goal of the design was to develop a system capable of recognizing an object and its present and near future location while also actuating a robotic arm in response to the motion of the ball in flight. The implementation made use of image processing functions in C++, NVIDIA Jetson TX1, Sterolabs’ ZED stereoscopic camera setup in connection to an embedded system controller for the robot arm. The image processing was done with a textured background and the 3D location coordinates were applied to the correction of a Kalman filter model that was used for estimating and predicting the ball location. A capture and processing speed of 59.4 frames per second was obtained with good accuracy in depth detection while the ball was well tracked in the tests carried out.