2D-3D registration is one of the underpinning technologies that enables image-guided intervention in computer-assisted orthopaedic surgery (CAOS). Preoperative 3D images and surgical plans need to be mapped to the patient in the operating room before they can be used to augment the surgical intervention, and this task is generally fulfilled by using 2D-3D registration which spatially aligns a preoperative 3D image to a set of intraoperative fluoroscopic images. The key problem in 2D-3D registration is to define an accurate similarity metric between the 2D and 3D data, and choose an appropriate optimization algorithm. Various similarity metrics and optimization algorithms have been proposed for 2D-3D registration; however, current techniques have several critical limitations. First, a good initial guess - usually within a few millimetres from the true solution - is required, and such capture range is often not wide enough for clinical use. Second, for currently used optimization algorithms, it is difficult to achieve a good balance between the computation efficiency and registration accuracy. Third, most current techniques register a 3D image of a single bone to a set of fluoroscopic images, but in many CAOS procedures, such as a multi-fragment fracture treatment, multiple bone pieces are involved. In this thesis, research has been conducted to investigate the above problems: 1) two new registration techniques are proposed that use recently developed optimization techniques, i.e. Unscented Kalman Filter (UKF) and Covariance Matrix Adaptation Evolution Strategy (CMA-ES), to improve the capture range for the 2D-3D registration problem; 2) a multiple-object 2D-3D registration technique is proposed that simultaneously aligns multiple 3D images of fracture fragments to a set of fluoroscopic images of fracture ensemble; 3) a new method is developed for fast and efficient construction of anatomical atlases; and 4) a new atlas-based multiple-object 2D-3D registration technique is proposed to aid fracture reduction in the absence of preoperative 3D images. Experimental results showed that: 1) by using the new optimization algorithms, the robustness against noise and outliers was improved, and the registrations could be performed more efficiently; 2) the simultaneous registration of multiple bone fragments could achieve a clinically acceptable global alignment among all objects with reasonable computation cost; and 3) the new atlas construction method could construct and update intensity atlases accurately and efficiently; and 4) the use of atlas in multiple-object 2D-3D registration is feasible. / Thesis (Ph.D, Computing) -- Queen's University, 2011-09-28 10:58:04.406

The first part of this work presents an accurate analysis of the most relevant 3D registration techniques, including initial pose estimation, pairwise registration and multiview registration strategies. A new classification has been proposed, based on both the applications and the approach of the methods that have been discussed.The main contribution of this thesis is the proposal of a new 3D multiview registration strategy. The proposed approach detects revisited regions obtaining cycles of views that are used to reduce the inaccuracies that may exist in the final model due to error propagation. The method takes advantage of both global and local information of the registration process, using graph theory techniques in order correlate multiple views and minimize the propagated error by registering the views in an optimal way. The proposed method has been tested using both synthetic and real data, in order to show and study its behavior and demonstrate its reliability. / La primera part d'aquest treball presenta una anàlisi acurada de les tècniques de registre 3D es rellevants, incloent tècniques d'estimació de la posició inicial, registre pairwise i registre entre múltiples vistes. S'ha proposat una nova classificació de les tècniques, depenent de les seves aplicacions i de l'estratègia utilitzada.La contribució mes important d'aquesta tesi és la proposta d'un nou mètode de registre 3D utilitzant múltiples vistes. El mètode proposat detecta regions ja visitades prèviament, obtenint cicles de vistes que s'utilitzen per tal de reduir els desalineaments en el model final deguts principalment a la propagació de l'error durant el procés de registre. Aquest mètode utilitza tant informació global com local, correlacionant les vistes mitjançant tècniques de grafs que permeten minimitzar l'error propagat i registrar les vistes de forma òptima. El mètode proposat ha estat provat utilitzant dades sintètiques i reals, per tal de mostrar i analitzar el seu comportament i demostrar la seva eficàcia.

Object recognition is one of the most important problems in computer vision. Traditional object recognition techniques are usually performed on optical images that are 2D projections of the 3D world. Information about the depth of objects in the scene is not provided explicitly in these images and thus, it makes 2D object recognition techniques sensitive to changes in illumination and shadowing. As surface acquisition methods such as LADAR or range scanners are becoming more popular, there is an increasing interest in the use of three-dimensional geometric data in object recognition to overcome these limitations. However, the matching of 3D free-form surfaces is also a difficult problem due to the shape and topological complexity of 3D surfaces. In addition, the problem is further complicated by other issues such as variations in surface sampling resolution, occlusion, clutter and sensor noise. The huge amount of information required to describe a 3D surface is also another challenge that 3D surface matching techniques have to deal with. This thesis investigates the problems of 3D surface matching that include 3D surface registration and object recognition from range images. It focuses on developing a novel and efficient framework for aligning 3D surfaces in different coordinate systems and from this, recognizing 3D models from scenes with high levels of occlusion and clutter using multi-scale local features. The first part of the thesis presents two different schemes for extracting salient geometric features from 3D surfaces using surface curvature measures known as the curvedness and shape index. By deriving the scale-space representation of the input surface, surface positions with high local curvature or high local shape variations are selected as features at various degrees of scale. One advantage of the proposed approaches is their applicability to both 3D meshes with connectivity information and unstructured point clouds. In the second part of the thesis, an application of the multi-scale feature extraction framework to 3D surface registration and object recognition is proposed. A Delaunay tetrahedrization is performed on the features extracted from each input range image to obtain a set of triangles. Possible correspondences are found by matching all possible pairs of triangles between the scene and model surfaces. From these correspondences, possible transformations between the two surfaces can be hypothesized and tested. In order to increase the accuracy and efficiency of the algorithm, various surface geometric and rigidity constraints are applied to prune unlikely correspondences. By finding the match that aligns the largest number of features between the two surfaces, the best transformation can be estimated. In the case of surface registration, this transformation can be used to coarse-align two different views of the same object. In the case of 3D object recognition, it provides information about the possible pose (location and orientation) of the model in the scene surface. Experimental results on a variety of 3D models and real scenes are shown to verify the effectiveness and robustness of the approach. / http://proxy.library.adelaide.edu.au/login?url= http://library.adelaide.edu.au/cgi-bin/Pwebrecon.cgi?BBID=1474505 / Thesis (M.App.Sc.) -- University of Adelaide, School of Electrical and Electronic Engineering, 2009

Accurate registration of 3D data is one of the most challenging problems in a number of Computer Vision applications. Visual Odometry is one such application, which determines the motion, or change in position of a moving rover by registering 3D data captured by an on-board range sensor, in a pairwise manner. The performance of Visual Odometry depends upon two main factors, the first being the quality of 3D data, which itself depends upon the type of sensor being used. The second factor is the robustness of the registration algorithm. Where sensors like stereo cameras and LIDAR scanners have been used in the past to improve the performance of Visual Odometry, the introduction of the Velodyne LIDAR scanner is fairly new and has been less investigated, particularly for odometry applications. This thesis presents and examines a new method for registering 3D point clouds generated by a Velodyne scanner mounted on a moving rover. The method is based on one of the the most widely used registration algorithms called Iterative Closest Point (ICP). The proposed method is divided into two steps. The first step, which is also the main contribution of this work, is the introduction of a new point sampling method, which prudently select points that belong to the regions of greatest geometric variance in the scan. Interest Point (Region) Sampling plays an important role in the performance of ICP by effectively discounting the regions with non-uniform resolution and selecting regions with a high geometric variance and uniform resolution. Second step is to use sampled scan pairs as the input to a new plane-to-plane variant of ICP, known as Generalized ICP. Several experiments have been executed to test the compatibility and robustness of Interest Point Sampling (IPS) for a variety of terrain landscapes. Through these experiments, which include comparisons of variants of ICP and past sampling methods, this work demonstrates that the combination of IPS and GICP results in the least localization error as compared to all other tested method. / Thesis (Master, Electrical & Computer Engineering) -- Queen's University, 2011-11-03 11:12:43.596

<p>The typical High Dose Rate (HDR) lung brachytherapy procedure involves inserting treatment catheters into the bronchi next to the tumour location using a bronchoscope. The anterior-posterior and lateral fluoroscopy images are acquired in order to localize the catheters prior to treatment. Although, these images enable accurate reconstruction of the catheter location, they do not allow for the visualization of the tumour or organs-at-risk due to poor soft tissue contrast. Although CT images offer an improved soft tissue contrast, moving the patient with catheters in place prior to each treatment is impractical.</p> <p>An alternative option is to use prior diagnostic or external beam radiation treatment planning CT images. These images cannot be used for treatment planning directly because of the variation in patient positioning between the CT and orthogonal images acquisition. In order to account for positioning differences, a 2D/3D registration algorithm that registers the orthogonal images with a previously acquired CT data was developed. The algorithm utilizes a rigid registration model based on a pixel/voxel intensity matching approach. A similarity measure combining normalized mutual information (NMI), image gradient, and intensity difference was developed. Evaluation of the algorithm was performed using tissue equivalent phantoms, and, in the clinical setting using data from six patients. The mean registration error was 2.1 mm and 3.2 mm for phantoms and patients respectively.</p> <p>External objects such as the treatment table and ECG leads are often in CT images, affecting the above mentioned 2D/3D registration. To address this, an algorithm for automatic removal of external objects from CT images was developed. This was applied to automatic contouring and removal of the fiducial markers in CT images used for external beam radiation therapy treatment planning for breast cancer. The algorithm was further modified to compute the girth of patients as part of a diagnostic radiology clinical trial.</p> / Doctor of Philosophy (PhD)

While robots are extensively used in factories, our industry hasn't yet been able to prepare them for working in human environments - for instance in houses or in human-operated factories. The main obstacle to these applications lies in the amplitude of the uncertainty inherent to the environments humans are used to work in, and in the difficulty in programming robots to cope with it. For instance, in robot-oriented environments, robots can expect to find specific tools and objects in specific places. In a human environment, obstacles may force one to find a new way of holding a tool, and new objects appear continuously and need to be dealt with. As it proves difficult to build into robots the knowledge necessary for coping with uncertain environments, the robotics community is turning to the development of agents that acquire this knowledge progressively and that adapt to unexpected events. This thesis studies the problem of vision-based robotic grasping in uncertain environments. We aim to create an autonomous agent that develops grasping skills from experience, by interacting with objects and with other agents. To this end, we present a 3D object model for autonomous, visuomotor interaction. The model represents grasping strategies along with visual features that predict their applicability. It provides a robot with the ability to compute grasp parameters from visual observations. The agent acquires models interactively by manipulating objects, possibly imitating a teacher. With time, it becomes increasingly efficient at inferring grasps from visual evidence. This behavior relies on (1) a grasp model representing relative object-gripper configurations and their feasibility, and (2) a model of visual object structure, which aligns the grasp model to arbitrary object poses (3D positions and orientations). The visual model represents object edges or object faces in 3D by probabilistically encoding the spatial distribution of small segments of object edges or the distribution of small patches of object surface. A model is learned from a few segmented 3D scans or stereo images of an object. Monte Carlo simulation provides robust estimates of the object's 3D position and orientation in cluttered scenes. The grasp model represents the likelihood of success of relative object-gripper configurations. Initial models are acquired from visual cues or by observing a teacher. Models are then refined autonomously by ``playing' with objects and observing the effects of exploratory grasps. After the robot has learned a few object models, learning becomes a combination of cross-object generalization and interactive experience: grasping strategies are generalized across objects that share similar visual substructures; they are then adapted to new objects through autonomous exploration. The applicability of our model is supported by numerous examples of pose estimates in cluttered scenes, and by a robot platform that shows increasing grasping capabilities as it explores its environment.

No description available.

Le but de ce travail est d'analyser des scènes sous-marines naturelles et en particulier cartographier des environnements sous-marins en 3D. Il existe aujourd'hui de nombreuses techniques pour résoudre ce problème. L'originalité de ce travail se trouve dans la fusion de deux cartes obtenues avec des capteurs de différentes résolutions. Dans un premier temps, un engin autonome (ou un bateau) analyse les fonds marins avec un sonar multifaisceaux et crée une première carte globale de la zone. Cette carte est ensuite décomposée en petites cellules représentant une mosaïque du fond marin. Une deuxième analyse est ensuite réalisée sur certaines cellules particulières à l'aide d'un second capteur avec une résolution plus élevée. Cela permettra d'obtenir une carte détaillée 3D de la cellule. Un véhicule autonome sous-marin ou un plongeur muni d'un système de vision stéréoscopique effectuera cette acquisition.Ce projet se décompose en deux parties, la première s'intéressera à la reconstruction 3D de scènes sous-marines en milieu contraint à l'aide d'une paire stéréoscopique. La deuxième partie de l'étude portera sur l'aspect multimodal. Dans notre cas, nous utilisons cette méthode pour obtenir des reconstructions précises d'objets d'intérêts archéologiques (statues, amphores, etc.) détectés sur la carte globale.La première partie du travail concerne la reconstruction 3D de la scène sous-marine. Même si aujourd'hui le monde de la vision a permis de mieux appréhender ce type d'image, l'étude de scène sous-marine naturelle pose encore de nombreux problèmes. Nous avons pris en compte les bruits sous-marins lors de la création du modèle 3D vidéo ainsi que lors de la calibration des appareils photos. Une étude de robustesse à ces bruits a été réalisée sur deux méthodes de détections et d'appariements de points d'intérêt. Cela a permis d'obtenir des points caractéristiques précis et robustes pour le modèle 3D. La géométrie épipolaire nous a permis de projeter ces points en 3D. La texture a été ajoutée sur les surfaces obtenues par triangulation de Delaunay.La deuxième partie consiste à fusionner le modèle 3D obtenu précédemment et la carte acoustique. Dans un premier temps, afin d'aligner les deux modèles 3D (le modèle vidéo et le modèle acoustique), nous appliquons un recalage approximatif en sélectionnant manuellement quelques paires de points équivalents sur les deux nuages de points. Pour augmenter la précision de ce dernier, nous utilisons un algorithme ICP (Iterative Closest Points).Dans ce travail nous avons créé une carte 3D sous-marine multimodale réalisée à l'aide de modèles 3D « vidéo » et d'une carte acoustique globale. / This work aims to analyze natural underwater scenes and it focuses on mapping underwater environment in 3D. Today, many methods exist to solve this problem. The originality of this work lies in the fusion of two maps obtained from sensors of different resolutions. Initially, an autonomous vehicle (or boat) analyzes the seabed with multibeam sonar and creates a first global map of the area. This map is then divided into small cells representing a mosaic of the seabed. A second analysis is then performed on some particular cells using a second sensor with a higher resolution. This will provide a detailed map of the 3D cell. An autonomous underwater vehicle (AUV) or a diver with a stereoscopic vision system will make this acquisition. This project is divided into two parts; the first one focuses on the 3D reconstruction of underwater scenes in constrained environment using a stereoscopic pair. The second part investigates the multimodal aspect. In our study, we want to use this method to obtain accurate reconstructions of archaeological objects (statues, amphorae, etc.) detected on the global map. The first part of the work relates the 3D reconstruction of the underwater scene. Even if today the vision community has led to a better understanding of this type of images, the study of natural underwater scenes still poses many problems. We have taken into account the underwater noise during the creation of the 3D video model and during the calibration of cameras. A study of the noise robustness was performed on two methods of detection and matching of features points. This resulted into obtaining accurate and robust feature points for the 3D model. Epipolar geometry allowed us to project these points in 3D. The texture was added to the surfaces obtained by Delaunay triangulation.The second part consists of fusing the 3D model obtained previously with the acoustic map. To align the two 3D models (video and acoustic model), we use a first approximated registration by selecting manually few points on each cloud. To increase the accuracy of this registration, we use an algorithm ICP (Iterative Closest Point).In this work we created a 3D underwater multimodal map performed using 3D video model and an acoustic global map.

En aplicaciones de captura robotizada se ha hecho necesario el uso de información tridimensional de los objetos que son manipulados. Esta información puede obtenerse mediante dispositivos de adquisición 3D, tales como escáneres láser o cámaras de tiempo de vuelo, que proporcionan imágenes de rango de los objetos. En este trabajo de tesis se presenta un nuevo enfoque para encontrar, sin disponer de una estimación previa, la transformación rígida que produzca una alineación adecuada de las nubes de puntos obtenidas con esos dispositivos. El algoritmo realiza una búsqueda iterativa de correspondencias mediante la comparación de descriptores 2D en varios niveles de resolución utilizando para ello una medida de similitud específicamente diseñada para el descriptor propuesto en esta tesis. Este algoritmo de alineamiento se puede utilizar tanto para modelado 3D como para aplicaciones de manipulación de objetos en situaciones en las que los objetos estén parcialmente ocluidos o presenten simetrías. / In applications of robotic manipulation of objects, the use of three-dimensional information of objects being manipulated has been made necessary. This information can be obtained by 3D acquisition devices, such as laser scanners or cameras of flight time, providing range images of objects. This thesis presents a new approach to find, without having a previous estimate, the Euclidean transformation that produces a proper alignment of point clouds obtained with these devices. The algorithm performs an iterative search for correspondences by comparing 2D descriptors at various levels of resolution using a similarity measure specifically designed for the descriptor proposed in this thesis. This alignment algorithm can be used for both 3D modelling and robotic manipulation applications when objects are partially occluded or have symmetries.

Le cancer de la prostate est le cancer le plus fréquent chez l'homme en Europe, néanmoins il n'existe actuellement pas de technique d'imagerie permettant de détecter avec précision les tumeurs dans la glande. Sachant que les coupes histologiques contiennent la réalité de terrain concernant le diagnostic, il est nécessaire de recaler les images de chaque technique d'imagerie aux coupes histologiques afin de pouvoir les évaluer. De plus, comme il n'existe pas de méthode permettant de contrôler précisément le plan de coupe histologique, le recalage doit être considéré comme un problème 3D. Un dispositif permettant de réaliser, de manière rapide et standardisée, des marqueurs internes dans les coupes histologiques a été développé, de même qu'un algorithme permettant de détecter automatiquement ces marqueurs, de les identifier et d'aligner les coupes histologiques. La méthode a été testée sur 10 prostates, avec en moyenne 19.2 coupes par prostate, et a permis d'obtenir une précision de recalage moyenne de 0.18 ± 0.13 mm au niveau des marqueurs. Un deuxième algorithme a été développé pour recaler les coupes histologiques, une fois alignées, avec les images IRM. Ce recalage a été conçu pour être guidé par les canaux éjaculateurs, un repère anatomique présent dans chaque prostate et visible à la fois en histologie et dans les images IRM cliniques, acquises avec une résolution standard. L'algorithme a d'abord été testé en s'appuyant sur les marqueurs artificiels. La précision obtenue pour le recalage était en moyenne de 0.45±0.25 mm au niveau des marqueurs et de 1.04 ± 0.21 mm au niveau des canaux éjaculateurs. L'algorithme a enfin été testé en guidant le recalage à l'aide de la position des canaux éjaculateurs. La précision moyenne obtenue était alors de 0.16±0.05 mm au niveau des canaux éjaculateurs et de 2.82±0.41 mm au niveau des marqueurs. Ces résultats suggèrent une valeur du facteur de rétrécissement de l'ordre de 1.07±0.03 et une inclinaison vis à vis du plan de coupe histologique de l'ordre de 13.6◦ ± 9.61◦, avec une variance importante pour ces deux paramètres / Prostate cancer is the most frequently diagnosed cancer of men in Europe, yet no current imaging technique is capable of detecting with precision tumours in the prostate. The histology slices are the gold standard for the diagnosis. Therefore, in order to evaluate each imaging technique, the histology slices must be precisely registered to the imaged data. As it cannot be assumed that the histology slices are cut along the same plane as the imaged data is acquired, the registration must be considered as a 3D problem. An apparatus has been developed that enables internal fiducial markers to be created in the histology slices in a rapid and standardised manner. An algorithm has been developed that automatically detects and identifies these markers, enabling the alignment of the histology slices. The method has been tested on 10 prostate specimens, with 19.2 slices on average per specimen. The accuracy of the alignment at the fiducial markers was on average 0.18±0.13 mm. A second algorithm was developed to 3D register the aligned histology slices with the MR images. The registration is designed to be guided by the ejaculatory ducts, an anatomical landmark present in every prostate and visible in both histology and MR images acquired at standard clinical resolution. The algorithm was first tested by using the fiducial needles to guide the registration. The average registration accuracy was 0.45 ± 0.25 mm at the fiducial needles and 1.04±0.21 mm at the ejaculatory ducts. The algorithm was then tested by using the ejaculatory ducts to guide the registration. The average registration accuracy was 0.16±0.05 mm at the ejaculatory ducts and 2.82 ± 0.41 mm at the fiducial needles. The results suggest that the histology shrinkage factor is of the order 1.07±0.03 and the tilt of the histology slicing plane is 13.6◦ ±9.61◦, with both parameters showing significant variance