2D-3D registration is one of the underpinning technologies that enables image-guided intervention in computer-assisted orthopaedic surgery (CAOS). Preoperative 3D images and surgical plans need to be mapped to the patient in the operating room before they can be used to augment the surgical intervention, and this task is generally fulfilled by using 2D-3D registration which spatially aligns a preoperative 3D image to a set of intraoperative fluoroscopic images. The key problem in 2D-3D registration is to define an accurate similarity metric between the 2D and 3D data, and choose an appropriate optimization algorithm. Various similarity metrics and optimization algorithms have been proposed for 2D-3D registration; however, current techniques have several critical limitations. First, a good initial guess - usually within a few millimetres from the true solution - is required, and such capture range is often not wide enough for clinical use. Second, for currently used optimization algorithms, it is difficult to achieve a good balance between the computation efficiency and registration accuracy. Third, most current techniques register a 3D image of a single bone to a set of fluoroscopic images, but in many CAOS procedures, such as a multi-fragment fracture treatment, multiple bone pieces are involved. In this thesis, research has been conducted to investigate the above problems: 1) two new registration techniques are proposed that use recently developed optimization techniques, i.e. Unscented Kalman Filter (UKF) and Covariance Matrix Adaptation Evolution Strategy (CMA-ES), to improve the capture range for the 2D-3D registration problem; 2) a multiple-object 2D-3D registration technique is proposed that simultaneously aligns multiple 3D images of fracture fragments to a set of fluoroscopic images of fracture ensemble; 3) a new method is developed for fast and efficient construction of anatomical atlases; and 4) a new atlas-based multiple-object 2D-3D registration technique is proposed to aid fracture reduction in the absence of preoperative 3D images. Experimental results showed that: 1) by using the new optimization algorithms, the robustness against noise and outliers was improved, and the registrations could be performed more efficiently; 2) the simultaneous registration of multiple bone fragments could achieve a clinically acceptable global alignment among all objects with reasonable computation cost; and 3) the new atlas construction method could construct and update intensity atlases accurately and efficiently; and 4) the use of atlas in multiple-object 2D-3D registration is feasible. / Thesis (Ph.D, Computing) -- Queen's University, 2011-09-28 10:58:04.406

<p>The typical High Dose Rate (HDR) lung brachytherapy procedure involves inserting treatment catheters into the bronchi next to the tumour location using a bronchoscope. The anterior-posterior and lateral fluoroscopy images are acquired in order to localize the catheters prior to treatment. Although, these images enable accurate reconstruction of the catheter location, they do not allow for the visualization of the tumour or organs-at-risk due to poor soft tissue contrast. Although CT images offer an improved soft tissue contrast, moving the patient with catheters in place prior to each treatment is impractical.</p> <p>An alternative option is to use prior diagnostic or external beam radiation treatment planning CT images. These images cannot be used for treatment planning directly because of the variation in patient positioning between the CT and orthogonal images acquisition. In order to account for positioning differences, a 2D/3D registration algorithm that registers the orthogonal images with a previously acquired CT data was developed. The algorithm utilizes a rigid registration model based on a pixel/voxel intensity matching approach. A similarity measure combining normalized mutual information (NMI), image gradient, and intensity difference was developed. Evaluation of the algorithm was performed using tissue equivalent phantoms, and, in the clinical setting using data from six patients. The mean registration error was 2.1 mm and 3.2 mm for phantoms and patients respectively.</p> <p>External objects such as the treatment table and ECG leads are often in CT images, affecting the above mentioned 2D/3D registration. To address this, an algorithm for automatic removal of external objects from CT images was developed. This was applied to automatic contouring and removal of the fiducial markers in CT images used for external beam radiation therapy treatment planning for breast cancer. The algorithm was further modified to compute the girth of patients as part of a diagnostic radiology clinical trial.</p> / Doctor of Philosophy (PhD)

Ette thèse s’inscrit dans le contexte de la réalité augmentée (RA). La problématique majeure consiste à calculer la pose d’une caméra en temps réel. Ce calcul doit être effectué en respectant trois critères principaux : précision, robustesse et rapidité. Dans le cadre de cette thèse, nous introduisons certaines méthodes permettant d’exploiter au mieux les primitives des images. Dans notre cas, les primitives sont des points que nous allons détecter puis décrire dans une image. Pour ce faire, nous nous basons sur la texture de cette image. Nous avons dans un premier temps mis en place une architecture favorisant le calcul rapide de la pose, sans perdre en précision ni en robustesse. Nous avons pour cela exploité une phase hors ligne, où nous reconstruisons la scène en 3D. Nous exploitons les informations que nous obtenons lors de cette phase hors ligne afin de construire un arbre de voisinage. Cet arbre lie les images de la base de données entre elles. Disposer de cet arbre nous permet de calculer la pose de la caméra plus efficacement en choisissant les images de la base de données jugées les plus pertinentes. Nous rendant compte que la phase de description et de comparaison des primitives n’est pas suffisamment rapide, nous en avons optimisé les calculs. Cela nous a mené jusqu’à proposer notre propre descripteur. Pour cela, nous avons dressé un schéma générique basé sur la théorie de l’information qui englobe une bonne part des descripteurs binaires, y compris un descripteur récent nommé BOLD [BTM15]. Notre objectif a été, comme pour BOLD, d’augmenter la stabilité aux changements d’orientation du descripteur produit. Afin de réaliser cela, nous avons construit un nouveau schéma de sélection hors ligne plus adapté à la procédure de mise en correspondance en ligne. Cela permet d’intégrer ces améliorations dans le descripteur que nous construisons. Procéder ainsi permet d’améliorer les performances du descripteur notamment en terme de rapidité en comparaison avec les descripteurs de l’état de l’art. Nous détaillons dans cette thèse les différentes méthodes que nous avons mises en place afin d’optimiser l’estimation de la pose d’une caméra. Nos travaux ont fait l’objet de 2 publications (1 nationale et 1 internationale) et d’un dépôt de brevet. / This thesis belongs within the context of augmented reality. The main issue resides in estimating a camera pose in real-time. This estimation should be done following three main criteria: precision, robustness and computation efficiency.In the frame of this thesis we established methods enabling better use of image primitives. As far as we are concerned, we limit ourselves to keypoint primitives. We first set an architecture enabling faster pose estimation without loss of precision or robustness. This architecture is based on using data collected during an offline phase. This offline phase is used to construct a 3D point cloud of the scene. We use those data in order to build a neighbourhood graph within the images in the database. This neighbourhood graph enables us to select the most relevant images in order to compute the camera pose more efficiently. Since the description and matching processes are not fast enough with SIFT descriptor, we decided to optimise the bottleneck parts of the whole pipeline. It led us to propose our own descriptor. Towards this aim, we built a framework encompassing most recent binary descriptors including a recent state-of-the-art one named BOLD. We pursue a similar goal to BOLD, namely to increase the stability of the produced descriptors with respect to rotations. To achieve this goal, we have designed a novel offline selection criterion which is better adapted to the online matching procedure introduced in BOLD.In this thesis we introduce several methods used to estimate camera poses more efficiently. Our work has been distinguished by two publications (a national and an international one) as well as with a patent application.

Cette thèse s'intéresse au recalage de données issues de capteurs 3D et multispectraux pour l'étude du patrimoine.Lorsque l'on étudie ce type d'objet, il y a souvent peu de points saillants naturels entre ces jeux de données complémentaires. Par ailleurs, l'utilisation de mires optiques est proscrite.Notre problème est donc de recaler des données multimodales sans points caractéristiques.Nous avons développé une méthode de recalage basé sur le suivi des systèmes d'acquisition en utilisant des techniques issues de la photogrammétrie.Des simulations nous ont permis d'évaluer la précision de la méthode dans trois configurations qui représentent des cas typiques dans l'étude d'objets du patrimoine.Ces simulations ont montré que l'on peut atteindre une précision du suivi de 0.020 mm spatialement et 0.100 mrad angulairement en utilisant quatre caméras 5 Mpx lorsque l'on numérise une zone de 400 mm x 700 mm.La précision finale du recalage repose sur le succès d'une série de calibrations optiques et géométriques, ainsi que sur leur stabilité pour la durée du processus d'acquisition.Plusieurs tests ont permis d'évaluer la précision du suivi et du recalage de plusieurs jeux de données indépendants; d'abord seulement 3D, puis 3D et multispecrales.Enfin, nous avons étendu notre méthode d'estimation de la réflectance à partir des données multispectrales lorsque celles-ci sont recalées sur un modèle 3D. / The concern and interest of this PhD thesis is the registration of featureless 3D and multispectral datasets describing cultural heritage objects.In this context, there are few natural salient features between the complementary datasets, and the use of targets is generally proscribed.We thus develop a technique based on the photogrammetric tracking of the acquisition systems in use.A series of simulations was performed to evaluate the accuracy of our method in three configurations chosen to represent a variety of cultural heritage objects.These simulations show that we can achieve a spatial tracking accuracy of 0.020 mm and an angular accuracy of 0.100 mrad using four 5 Mpx cameras when digitizing an area of 400 mm x 700 mm. The accuracy of the final registration relies on the success of a series of optical and geometrical calibrations and their stability for the duration of the full acquisition process.The accuracy of the tracking and registration was extensively tested in laboratory settings. We first evaluated the potential for multiview 3D registration. Then, the method was used for to project of multispectral images on 3D models.Finally, we used the registered data to improve the reflectance estimation from the multispectral datasets