Apesar dos avanços recentes na tecnologia dos aparelhos de ressonância magnética (RM) permitirem a aquisição de imagens de alta resolução, ainda não é possível delinear de forma confiável os limites entre regiões de diferentes citoarquiteturas baseando-se somente nesta modalidade. As imagens de histologia são mandatórias quando se necessita saber o limite exato entre diferentes regiões neuroanatômicas. Contudo, o processamento histológico inevitavelmente causa grandes deforma¸coes no tecido, o que torna a compara- ção direta entre as duas modalidades inviável. Os estudos de neuroimagem/neuroanatomia que necessitam de comparação com a histologia devem necessariamente incluir uma etapa de alinhamento entre as duas modalidades; tarefa que muitas vezes acaba sendo realizada manualmente. Entretanto, o registro manual ´e demorado e pouco acurado, se tornando inviável quando os exames de histologia geram centenas de imagens. Este trabalho propõe um método para registro de imagens de histologia e RM, composto por um conjunto de recomenda¸coes para o preparo das imagens cujo objetivo ´e otimizá-las para o registro; e por uma pipeline computacional capaz de registrar as imagens consideradas. O trabalho aqui descrito foi desenvolvido primeiramente com o intuito de registrar imagens de espécimens de hipocampo provenientes do projeto CINAPCE e, posteriormente, para registro de imagens de encéfalo inteiro provenientes do Banco de Cérebros da Faculdade de Medicina da Universidade de São Paulo. A pipeline computacional foi testada com sucesso em imagens reais de dois encéfalos inteiros. A avaliação quantitativa dos registros realizados foi feita comparando segmenta¸coes manuais do hipocampo direito, núcleo caudado esquerdo e ventrículos laterais superiores, realizadas no volume de RM e da histologia registrada. A quantificação do resultado foi feita através do cálculo das métricas coeficiente de Dice (CSD) e distancia espectral ponderada (DEP) sobre as segmentações. A pipeline obteve um CSD médio de aproximadamente 0,77 e um DEP médio de aproximadamente 0,003. Os resultados mostraram que o método foi capaz de registrar as imagens de histologia nas respectivas imagens de RM exigindo interação mínima com o usuário. / Although latest advances in MRI technology have allowed the acquisition of higher resolution images, reliable delineation of cytoarchitectural boundaries is not yet possible based solely on that modality. Histological images are regularly required to locate the exact limits between neuroanatomical structures. Histological processing is nevertheless prone to cause a high amount of tissue distortion, which prevents direct comparison between the two modalities. Neuroimage/neuroanatomy studies that require direct comparison between histology am MRI must include a registration step. Such task is usually manually performed, but that becames infeasible for large histology volumes. Moreover, manual registration is time consuming and inaccurate. This thesis proposes a set of tissue processing recommendations aiming at optimizing the registration proccess, together with a computational pipeline for registering histology to MRI. The herein described work was initially designed to proccess hippocampi specimens from the CINAPCE project and posteriorly improved to process full brain images from the Brain Bank of the Brazilian Aging Brain Study Group. The pipeline was tested on two full brain histology volumes from the Brain Bank of the Brazilian Aging Brain Study Group. Results were assessed by comparison of manual segmentations of the left caudate nucleus, right hippocampus and superior lateral ventricles, performend on both MRIs and registered histology volumes. Quatitative evaluation was performed by computing the Dice coeficient (DC) and normalized weighted spectral distance (WESD) on the segmentations. The pipeline precessing yielded mean DC of 0.77 and mean WESD of 0.0033. The described method was able to sucessfuly register histology to their corresponding MRI volumes with minimal user interaction.

Este trabalho descreve uma técnica para alinhamento de imagens de documentos antigos. Para que seja possível alinhar duas imagens da mesma página de um documento uma técnica de registro de imagens é utilizada. As imagens mencionadas possuem diferenças de aquisição que incluem translação, rotação e distorções geométricas. A técnica de registro de imagens é dividida em três etapas. A primeira é a seleção de pontos de controle. Nessa etapa, as imagens das páginas são segmentadas em linhas de texto e, em seguida, cada linha de texto é segmentada em palavras. Dessa forma, cada ponto de controle corresponde ao início de uma palavra no texto. Na segunda etapa é realizada uma correspondência entre os pontos de controle extraídos das duas imagens. Essa etapa é necessária, uma vez que a seleção de pontos apresenta algumas falhas e os dois conjuntos de pontos de controle não são iguais. Por fim, na última etapa, a imagem destino é então mapeada utilizando funções Thin Plate Spline - TPS para que coincida com a imagem de referência. / This work describes a technique to anciente document images alignment. To be possible to align a couple of images of the same document pages, a image registration approach is used. These mentioned images have acquisition differences like translation, rotation and geometric distorsions. The image registration approach is divided in three stages. The first one is the control points selection. In this stage, images of the pages are segmented in text lines and then each one of them is segmented in words. Therefore, each selected control point corresponds to the begin of a text word. In the second stage, a matching is performed between extracted control points of both images. This stage is quite necessary because points selection presents some faults and both control points sets are not equals. Finally, in the last stage, target image is mapped using thin plate spline functions to coincide it with the reference image.

Image guided surgery (IGS), which has been developing fast recently, benefits significantly from the superior accuracy of robots and magnetic resonance imaging (MRI) which is a great soft tissue imaging modality. Teleoperation is especially desired in the MRI because of the highly constrained space inside the closed-bore MRI and the lack of haptic feedback with the fully autonomous robotic systems. It also very well maintains the human in the loop that significantly enhances safety. This dissertation describes the development of teleoperation approaches and implementation on an example system for MRI with details of different key components. The dissertation firstly describes the general teleoperation architecture with modular software and hardware components. The MRI-compatible robot controller, driving technology as well as the robot navigation and control software are introduced. As a crucial step to determine the robot location inside the MRI, two methods of registration and tracking are discussed. The first method utilizes the existing Z shaped fiducial frame design but with a newly developed multi-image registration method which has higher accuracy with a smaller fiducial frame. The second method is a new fiducial design with a cylindrical shaped frame which is especially suitable for registration and tracking for needles. Alongside, a single-image based algorithm is developed to not only reach higher accuracy but also run faster. In addition, performance enhanced fiducial frame is also studied by integrating self-resonant coils. A surgical master-slave teleoperation system for the application of percutaneous interventional procedures under continuous MRI guidance is presented. The slave robot is a piezoelectric-actuated needle insertion robot with fiber optic force sensor integrated. The master robot is a pneumatic-driven haptic device which not only controls the position of the slave robot, but also renders the force associated with needle placement interventions to the surgeon. Both of master and slave robots mechanical design, kinematics, force sensing and feedback technologies are discussed. Force and position tracking results of the master-slave robot are demonstrated to validate the tracking performance of the integrated system. MRI compatibility is evaluated extensively. Teleoperated needle steering is also demonstrated under live MR imaging. A control system of a clinical grade MRI-compatible parallel 4-DOF surgical manipulator for minimally invasive in-bore prostate percutaneous interventions through the patient’s perineum is discussed in the end. The proposed manipulator takes advantage of four sliders actuated by piezoelectric motors and incremental rotary encoders, which are compatible with the MRI environment. Two generations of optical limit switches are designed to provide better safety features for real clinical use. The performance of both generations of the limit switch is tested. MRI guided accuracy and MRI-compatibility of whole robotic system is also evaluated. Two clinical prostate biopsy cases have been conducted with this assistive robot.

Medical images have been used increasingly for diagnosis, treatment planning, monitoring disease processes, and other medical applications. A large variety of medical imaging modalities exists including CT, X-ray, MRI, Ultrasound, etc. Frequently a group of images need to be compared to one another and/or combined for research or cumulative purposes. In many medical studies, multiple images are acquired from subjects at different times or with different imaging modalities. Misalignment inevitably occurs, causing anatomical and/or functional feature shifts within the images. Computerized image registration (alignment) approaches can offer automatic and accurate image alignments without extensive user involvement and provide tools for visualizing combined images. This dissertation focuses on providing automatic image registration strategies. After a through review of existing image registration techniques, we identified two registration strategies that enhance the current field: (1) an automated rigid body and affine registration using voxel similarity measurements based on a sequential hybrid genetic algorithm, and (2) an automated deformable registration approach based upon a linear elastic finite element formulation. Both methods streamlined the registration process. They are completely automatic and require no user intervention. The proposed registration strategies were evaluated with numerous 2D and 3D MR images with a variety of tissue structures, orientations and dimensions. Multiple registration pathways were provided with guidelines for their applications. The sequential genetic algorithm mimics the pathway of an expert manually doing registration. Experiments demonstrated that the sequential genetic algorithm registration provides high alignment accuracy and is reliable for brain tissues. It avoids local minima/maxima traps of conventional optimization techniques, and does not require any preprocessing such as threshold, smoothing, segmentation, or definition of base points or edges. The elastic model was shown to be highly effective to accurately align areas of interest that are automatically extracted from the images, such as brains. Using a finite element method to get the displacement of each element node by applying a boundary mapping, this method provides an accurate image registration with excellent boundary alignment of each pair of slices and consequently align the entire volume automatically. This dissertation presented numerous volume alignments. Surface geometries were created directly from the aligned segmented images using the Multiple Material Marching Cubes algorithm. Using the proposed registration strategies, multiple subjects were aligned to a standard MRI reference, which is aligned to a segmented reference atlas. Consequently, multiple subjects are aligned to the segmented atlas and a full fMRI analysis is possible.

Airborne multi-sensor imaging is increasingly used to examine vegetation properties. The advantage of using multiple types of sensor is that each detects a different feature of the vegetation, so that collectively they provide a detailed understanding of the ecological pattern. Specifically, Light Detection And Ranging (LiDAR) devices produce detailed point clouds of where laser pulses have been backscattered from surfaces, giving information on vegetation structure; hyperspectral sensors measure reflectances within narrow wavebands, providing spectrally detailed information about the optical properties of targets; while aerial photographs provide high spatial-resolution imagery so that they can provide more feature details which cannot be identified from hyperspectral or LiDAR intensity images. Using a combination of these sensors, effective techniques can be developed for mapping species and inferring leaf physiological processes at ITC-level. Although multi-sensor approaches have revolutionised ecological research, their application in mapping individual tree crowns is limited by two major technical issues: (a) Multi-sensor imaging requires all images taken from different sensors to be co-aligned, but different sensor characteristics result in scale, rotation or translation mismatches between the images, making correction a pre-requisite of individual tree crown mapping; (b) reconstructing individual tree crowns from unstructured raw data space requires an accurate tree delineation algorithm. This thesis develops a schematic way to resolve these technical issues using the-state-of-the-art computer vision algorithms. A variational method, called NGF-Curv, was developed to co-align hyperspectral imagery, LiDAR and aerial photographs. NGF-Curv algorithm can deal with very complex topographic and lens distortions efficiently, thus improving the accuracy of co-alignment compared to established image registration methods for airborne data. A graph cut method, named MCNCP-RNC was developed to reconstruct individual tree crowns from fully integrated multi-sensor imagery. MCNCP-RNC is not influenced by interpolation artefacts because it detects trees in 3D, and it detects individual tree crowns using both hyperspectral imagery and LiDAR. Based on these algorithms, we developed a new workflow to detect species at pixel and ITC levels in a temperate deciduous forest in the UK. In addition, we modified the workflow to monitor physiological responses of two oak species with respect to environmental gradients in a Mediterranean woodland in Spain. The results show that our scheme can detect individual tree crowns, find species and monitor physiological responses of canopy leaves.

Este trabalho descreve uma técnica para alinhamento de imagens de documentos antigos. Para que seja possível alinhar duas imagens da mesma página de um documento uma técnica de registro de imagens é utilizada. As imagens mencionadas possuem diferenças de aquisição que incluem translação, rotação e distorções geométricas. A técnica de registro de imagens é dividida em três etapas. A primeira é a seleção de pontos de controle. Nessa etapa, as imagens das páginas são segmentadas em linhas de texto e, em seguida, cada linha de texto é segmentada em palavras. Dessa forma, cada ponto de controle corresponde ao início de uma palavra no texto. Na segunda etapa é realizada uma correspondência entre os pontos de controle extraídos das duas imagens. Essa etapa é necessária, uma vez que a seleção de pontos apresenta algumas falhas e os dois conjuntos de pontos de controle não são iguais. Por fim, na última etapa, a imagem destino é então mapeada utilizando funções Thin Plate Spline - TPS para que coincida com a imagem de referência. / This work describes a technique to anciente document images alignment. To be possible to align a couple of images of the same document pages, a image registration approach is used. These mentioned images have acquisition differences like translation, rotation and geometric distorsions. The image registration approach is divided in three stages. The first one is the control points selection. In this stage, images of the pages are segmented in text lines and then each one of them is segmented in words. Therefore, each selected control point corresponds to the begin of a text word. In the second stage, a matching is performed between extracted control points of both images. This stage is quite necessary because points selection presents some faults and both control points sets are not equals. Finally, in the last stage, target image is mapped using thin plate spline functions to coincide it with the reference image.

Résumé : Thèse de DoctoratL’importance de l’imagerie médicale comme élément principal dans plusieurs application médicales et diagnostiques de soin de santé est indispensable. L’intégration des données utiles obtenues des diverses images est vitale pour une analyse appropriée de l’information contenues dans ces images sous observation. Pour que ce processus d’intégration réussisse, une procédure appelée recalage d’image est nécessaire.Le but du recalage d’image consiste à aligner deux images afin de trouver une transformation géométrique qui place une des deux images dans la meilleure correspondance spatiale possible avec l’autre image en optimisant un critère de recalage. Les deux images sont dites image cible et image source. Les méthodes de recalage d’image consistent à avoir référencées par des points de contrôle. Ceci est suivi d’une transformation de recalage qui associe les deux images et d’une fonction définie sur base de la mesure de similarité qui a pour but de mesurer la valeur qualitative de proximité ou encore de degré de concordance entre l’image cible et l’image source. Enfin, un optimisateur qui cherche à trouver la transformation optimale au sein du champ de solution de la recherche, est appliqué.Cette recherche présente un algorithme automatique de recalage d’image dont le but est de résoudre le problème du recalage d’image à multiple modes sur une paire des clichés de tomographie par ordinateur (CT) faite sur les poumons. Dans cette méthode, une étude comparative entre la méthode classique d’optimisation par algorithme du gradient à pas fixe et celle de l’algorithme évolutionniste est menée. L’objectif de cette recherche est d’effectuer l’optimisation par des techniques de recalage d’image ainsi qu’évaluer la performance de ces mêmes techniques afin de doter les spécialistes du domaine médical d’une estimation de combien précis et robuste le processus de recalage serait. Les paires des clichés obtenues de la tomographie par ordinateur faite sur les poumons sont recalées en utilisant l’information mutuelle comme mesure de similarité, la transformation affine ainsi que l’interpolation linéaire. Un optimisateur qui recherche la transformation optimale au sein de l’espace de recherche est appliqué afin de minimiser la fonction économique (fonction objectif).Les études de détermination d’un modèle de transformation qui dépend des paramètres de transformation et de l’identification des mesures de similarité basée sur l’intensité du voxel ont été menées. En alignant la transformation avec les points de control, trois modèles de transformation sont comparés. La transformation affine produit la meilleure reconstitution d’image en comparaison aux transformations non réfléchissantes et projectives. Les résultats de cette recherche sont assez comparables à celles rapportées dans le challenge de recherche EMPIRE 10 et sont conformes à la fois aux principes théoriques aussi bien qu’aux applications pratiques.La contribution de cette recherche réside dans son potentiel à améliorer la compréhension scientifique du recalage d’image des organes anatomiques du corps humain. Cette recherche établie ainsi une base pour une recherche avancée sur l’évaluation de la performance des techniques de recalage et la validation des procédures sur d’autres types d’algorithmes et domaines d’application du recalage d’images comme la détection, la communication par satellite, l’ingénierie biomédicale, la robotique, les systèmes d'information géographique (SIG) et de localisation parmi tant d’autres / D’Tech Thesis SummaryThe importance of medical imaging as a core component of several medical application and healthcare diagnosis cannot be over emphasised. Integration of useful data acquired from different images is vital for proper analysis of information contained in the images under observation. For the integration process to be successful, a procedure referred to as image registration is necessary.The purpose of image registration is to align two images in order to find a geometric transformation that brings one image into the best possible spatial correspondence with another image by optimising a registration criterion. The two images are known as the target image and the source image. Image registration methods consist of having the two images referenced with control points. This is followed by a registration transformation that relates the two images and a similarity metric function that aims to measure the qualitative value of closeness or degree of fitness between the target image and the source image. Finally, an optimiser which seeks an optimal transformation inside the defined solution search space is performed.This research presents an automated image registration algorithm for solving multimodal image registration on lung Computer Tomography (CT) scan pairs, where a comparison between regular step gradient descent optimisation technique and evolutionary optimisation was investigated. The aim of this research is to carry out optimisation and performance evaluation of image registration techniques in order to provide medical specialists with estimation on how accurate and robust the registration process is. Lung CT scan pairs are registered using mutual information as a similarity measure, affine transformation and linear interpolation. In order to minimise the cost function, an optimiser, which seeks the optimal transformation inside the defined search space is applied.Determination of a transformation model that depends on transformation parameters and identification of similarity metric based on voxel intensity were carried out. By fitting transformation to control points, three transformation models were compared. Affine transformation produced the best recovered image when compared to non-reflective similarity and projective transformations. The results of this research compares well with documented results from EMPIRE 10 Challenge research and conforms to both theoretical principles as well as practical applications.The contribution of this research is its potential to increase the scientific understanding of image registration of anatomical body organs. It lays a basis for further research in performance evaluation of registration techniques and validation of procedures to other types of algorithms and image registration application areas, such as remote sensing, satellite communication, biomedical engineering, robotics, geographical information systems and mapping, among others

Image registration plays an important role in pulmonary image analysis. Accurate image registration is a challenging problem when the lungs have deformation with large distance. Registration results estimate the local tissue movement and are useful for studying lung mechanical quantities. In this thesis, we propose a new registration algorithm and a registration scheme to solve lung CT matching problems. Approaches to study lung functions are discussed and presented through a practical application. The overall objective of our project is to develop image registration techniques and analysis approaches to measure lung functions at high resolution. We design a nonrigid volumetric registration algorithm to catch lung motion from a pair of intrasubject CT images acquired at different inflation levels. This registration algorithm preserves both parenchymal tissue volume and vesselness measure, and is regularized by a linear elasticity cost. Validation methods for lung CT matching are introduced and used to evaluate the performance of different registration algorithms. Evaluation shows the feature-based vesselness constraint can efficiently improve the registration accuracy around lung boundaries and in the base lung region. Meanwhile, a new scheme to solve complex registration problem is introduced utilizing both surface and volumetric registration. The first step of this scheme is to register the boundaries of two images using surface registration. The resulting boundary displacements are extended to the entire ROI domains using the Element Free Galerkin Method (EFGM) based on weighted extended B-Splines (WEB-Splines). These displacement fields are used as initial conditions for the tissue volume– and vessel–preserving non-rigid registration over the object domain. Both B-Splines and WEB-Splines are used to parameterize the transformations. Our algorithms achieve high accuracy and provide reasonable lung function maps. The mean errors on landmarks, vessel locations, and fissure planes are on the order of 1 mm (sub-voxel level). Furthermore, we establish methods based on registration derived transformation to analyze mechanical quantities and measure regional lung function. The proposed registration method and lung function measurement are applied on a practical application to detect mechanical alternations in the lung following bronchoalveolar lavage, which achieves satisfactory results and demonstrates the applicability of our proposed approaches.

This thesis mainly focuses on proposing a 4D (three spatial dimensions plus time) tissue-volume preserving non-rigid image registration algorithm for pulmonary 4D computed tomography (4DCT) data sets to provide relevant information for radiation therapy and to estimate pulmonary ventilation. The sum of squared tissue volume difference (SSTVD) similarity cost takes into account the CT intensity changes of spatially corresponding voxels, which is caused by variations of the fraction of tissue within voxels throughout the respiratory cycle. The proposed 4D SSTVD registration scheme considers the entire dynamic 4D data set simultaneously, using both spatial and temporal information. We employed a uniform 4D cubic B-spline parametrization of the transform and a temporally extended linear elasticity regularization of deformation field to ensure temporal smoothness and thus biological plausibility of estimated deformation. A multi-resolution multi-grid registration framework was used with a limited-memory Broyden Fletcher Goldfarb Shanno (LBFGS) optimizer for rapid convergence rate, robustness against local minima and limited memory consumption. The algorithm was prototyped in Matlab and then fully implemented in C++ in Elastix package based on the Insight Segmentation and Registration Toolkit (ITK). We conducted experiments on 2D+t synthetic images to demonstrate the effectiveness of the proposed method. The 4D SSTVD algorithm was also tested on clinical pulmonary 4DCT data sets in comparison with existing 3D pairwise SSTVD algorithm and 4D sum of squared difference (SSD) algorithm. The mean landmark error and mean landmark irregularity were calculated based on manually annotated landmarks on publicly available 4DCT data sets to evaluate the accuracy and temporal smoothness of the registration results. A 4D landmarking software tool was also designed and implemented in Java as an ImageJ plug-in to help facilitate the landmark labeling process in 4DCT data sets.

Current radiation therapy (RT) planning for limiting lung toxicity is based on a uniform lung function with little consideration to the spatial and temporal pattern of lung function. Establishment of relationships between radiation dose and changes in pulmonary function can help predict and reduce the RT-induced pulmonary toxicity. Baseline measurement uncertainty of pulmonary function across scans needs to be assessed, and there is a great interest to compensate the pulmonary function for respiratory effort variations. Respiratory-gated 4DCT imaging and image registration can be used to estimate the regional lung volume change by a transformation-based ventilation metric which is computed directly from the deformation field, or a intensity-based metric which is based on CT density change in the registered image pair. In this thesis, we have evaluated the reproducibility of regional pulmonary function measures using two repeated 4D image acquisitions taken within a short time interval for both transformation-based and intensity-based metrics. Furthermore, we have proposed and compared normalization schemes that correct ventilation images for variations in respiratory effort and assess the reproducibility improvement after effort correction. The major contributions of this thesis include: 1) develop and validate a process for establishing measurement reproducibility in 4DCT-based ventilation, 2) evaluate reproducibility of the transformation-based ventilation measurement, 3) evaluate reproducibility of the intensity-based ventilation measurement, 4) develop and compare different ventilation normalization methods to correct for respiratory effort variation across scans.