Groupwise non-rigid image registration is a powerful tool to automatically establish correspondences across sets of images. Such correspondences are widely used for constructing statistical models of shape and appearance. As existing techniques usually treat registration as an optimisation problem, a good initialisation is required. Although the standard initialisation---affine transformation---generally works well, it is often inadequate when registering images of complex structures. In this thesis we present a sophisticated system that uses the sparse matches of one or more parts+geometry models as the initialisation. We show that both the model/s and its/their matches can be automatically obtained, and that the matches are able to effectively initialise a groupwise non-rigid registration algorithm, leading to accurate dense correspondences. We also show that the dense mesh models constructed during the groupwise registration process can be used to accurately annotate new images. We demonstrate the efficacy of the proposed system on three datasets of increasing difficulty, and report on a detailed quantitative evaluation of its performance.

Em radioterapia externa, uma nova técnica chamada terapia de radiação de intensidade modulada - IMRT - permite delinear a dose de radiação em imagens de 2 ou 3 dimensões, delimitando de forma bastante precisa e não necessariamente uniforme a região a ser irradiada. Assim, ao mesmo tempo que o tumor é irradiado, é possível evitar a irradiação aos tecidos vizinhos íntegros (sãos), limitando os efeitos secundários do tratamento. Para que a radioterapia externa tenha sucesso usando a técnica IMRT, é fundamental delinear previamente de forma precisa o tumor e os órgãos sãos que devem ser protegidos da radiação, garantindo assim a dose exata de radiação nos volumes alvos. O objetivo desta tese é fornecer ferramentas que sejam adequadas ao delineamento automático de estruturas de interesse e à radioterapia adaptativa para tumores da região da cabeça e do pescoço. Atualmente, a segmentação de estruturas de interesse, tais como os órgãos em risco e as regiões de propagação tumoral, é feita manualmente. Esta é uma tarefa que demanda bastante tempo de um especialista, além de ser tediosa. Além do mais, o planejamento em radioterapia é feito baseado na imagem adquirida na semana do pré-tratamento, onde é calculada a dose. Normalmente o tratamento ocorre em várias semanas, porém a dose estimada no início do tratamento é a mesma para todas as outras semanas do tratamento. Calcular a dose e mantê-la nas demais semanas é uma simplificação que não corresponde à realidade, já que ocorrem mudanças anatômicas no paciente ao longo do tratamento. Estas mudanças ocorrem devido ao encolhimento do tumor e ao possível emagrecimento do paciente, provocando alterações anatômicas locais e globais. As contribuições desta tese visam solucionar e avançar nestes problemas e são apresentadas em dois eixos. No primeiro eixo, é proposta uma metodologia para escolher uma anatomia que seja representativa da população, anatomia esta chamada de atlas. O registro do atlas na imagem do paciente permite que estruturas de interesse sejam segmentadas automaticamente, acelerando o processo de delineamento e tornando-o mais robusto. A segunda contribuição desta tese é voltada à radioterapia adaptativa. Para que a dose estimada na primeira semana seja adaptada às modificações anatômicas, é necessária a utilização de métodos de registro não-rígidos. Portanto, nesta etapa é feita uma avaliação e adaptação dos métodos de registros de forma que a região do tumor esteja bem alinhada. / Intensity Modulated Radiotherapy (IMRT) is a new technique enabling the delineation of the 3D radiation dose. It allows to delineate a radiation zone of almost any shape and to modulate the beam intensity inside the target. If IMRT enables to constrain the radiation plan in the beam delivery as well as in the protection of important functional areas (e.g. spinal cord), it also raises the issues of adequacy and accuracy of the selection and delineation of the target volumes. The purpose of this thesis is to provide tools to automatic delineation of the regions of interest and also to adaptive radiotherapy treatment for tumors located in the head and neck region. The delineation in the patient computed tomography image of the tumor volume and organs to be protected is currently performed by an expert who delineates slice by slice the contours of interest. This task is highly time-consuming and requires experts’ knowledge. Moreover, the planning process in radiotherapy typically involves the acquisition of a unique set of computed tomography images in treatment position on which target volumes (TVs) and normal structures are delineated, and which are used for dose calculation. Restricting the delineation of these regions of interest based solely on pre-treatment images is an oversimplification as it is only a snapshot of the patient´s anatomy at a given time. Shrinkage of the tumor and modification of the patient anatomy at large (e.g. due to weight loss) may indeed occur within the several weeks’ duration of a typical treatment. The main contributions of this thesis aim to advance in the solution to these issues and are presented in two axes. In the first one, it is proposed a methodology to choose an image with the most representative anatomy of a population; such image is called Atlas. The registration of the atlas into a new image of the patient allows to automatically segment the structures of interest, speeding up the delineation process and making it more robust. The second contribution of this thesis is focused on the adaptive radiotherapy. In order to adjust the estimated dose to the anatomical modifications, it is fundamental to have non-rigid registration algorithms. So, the evaluation and adaptation of non-rigid registration methods are required, addressing especially the alignment of the tumor’s region among different moments of the treatment.

Tensor scale (t-scale) is a parametric representation of local structure morphology that simultaneously describes its orientation, shape and isotropic scale. At any image location, t-scale is the parametric representation of the largest ellipse (an ellipsoid in 3D) centered at that location and contained in the same homogeneous region. Recently, we have improved the t-scale computation algorithm by: (1) optimizing digital representations for LoG and DoG kernels for edge detection and (2) ellipse fitting by using minimization of both algebraic and geometric distance errors. Also, t-scale has been applied to computing the deformation vector field with applications to medical image registration. Currently, the method is implemented in two-dimension (2D) and the deformation vector field is directly computed from t-scale-derived normal vectors at matching locations in two images to be registered. Also, the method has been used to develop a simple algorithm for computing 2D warping from one shape onto another. Meanwhile, t-scale has been applied to generating interpolation lines with applications to medical image interpolation using normal vector. Normal vector yields local structure orientation pointing to the closest edge. However, this information is less reliable along the medial axis of a shape as it may be associated with either of the two opposite edges of the local shape. This problem is overcome using a shape-linearity measure estimating relative changes in scale along the orthogonal direction. Preliminary results demonstrate the method's potential in estimating deformation between two images and interpolating between neighboring slices in a grey scale image.

Image registration is a useful technique to measure the change between two or more images. Lung CT image registration is widely used an non-invasive method to measure the lung function changes. Non-invasive lung function measurement accuracy highly depends on lung CT image registration accuracy. Improving the registration accuracy is an important issue. In this thesis, we propose incorporating information of the anatomical structure of the lung (fissures) as an additional cost function of the lung CT image registration. The intensity-based similarity measurement method (sum of the squared tissue volume differences) is also used to complement lung tissue information matching. However, since fissures are hard to segment, a sheet-likeness filter is applied to detect fissure-like structures. Sheet-likeness is used as an additional cost function of the intensity-based registration. The registration accuracy is verified by the visual assessment and landmark error measurement. The landmark error measurement can show an improvement of the proposed algorithm.

In current practice, it is common in medical diagnosis or treatment monitoring for a patient to require multiple examinations using different imaging techniques. Magnetic resonance (MR) imaging and computed tomography (CT) are good at providing anatomical information. Three-dimensional functional information about tissues and organs is often obtained with radionuclide imaging modalities: positron emission tomography (PET) and single photon emission tomography (SPET). In nuclear medicine, such techniques must contend with poor spatial resolution, poor counting statistics of functional images and the lack of correspondence between the distribution of the radioactive tracer and anatomical boundaries. Information gained from anatomical and functional images is usually of a complementary nature. Since the patient cannot be relied on to assume exactly the same pose at different times and possibly in different scanners, spatial alignment of images is needed. In this thesis, a general framework for image registration is presented, in which the optimum alignment corresponds to a maximum of a similarity measure. Particular attention is drawn to entropy-based measures, and variance-based measures. These similarity measures include mutual information, normalized mutual information and correlation ratio which are the ones being considered in this study. In multimodality image registration between functional and anatomical images, these measures manifest superior performance compared to feature-based measures. A common characteristic of these measures is the use of the joint-intensity histogram, which is needed to estimate the joint probability and the marginal probability of the images. A novel similarity measure is proposed, the symmetric correlation ratio (SCR), which is a simple extension of the correlation ratio measure. Experiments were performed to study questions pertaining to the optimization of the registration process. For example, do these measures produce similar registration accuracy in the non-brain region as in the brain? Does the performance of SPET-CT registration depend on the choice of the reconstruction method (FBP or OSEM)? The joint-intensity based similarity measures were examined and compared using clinical data with real distortions and digital phantoms with synthetic distortions. In automatic SPET-MR rigid-body registration applied to clinical brain data, a global mean accuracy of 3.9 mm was measured using external fiducial markers. SCR performed better than mutual information when sparse sampling was used to speed up the registration process. Using the Zubal phantom of the thoracic-abdominal region, SPET projections for Methylenediphosponate (MDP) and Gallium-67 (67Ga) studies were simulated for 360 degree data, accounting for noise, attenuation and depth-dependent resolution. Projection data were reconstructed using conventional filtered back projection (FBP) and accelerated maximum likelihood reconstruction based on the use of ordered subsets (OSEM). The results of SPET-CT rigid-body registration of the thoracic-abdominal region revealed that registration accuracy was insensitive to image noise, irrespective of which reconstruction method was used. The registration accuracy, to some extent, depended on which algorithm (OSEM or FBP) was used for SPET reconstruction. It was found that, for roughly noise-equivalent images, OSEM-reconstructed SPET produced better registration than FBP-reconstructed SPET when attenuation compensation (AC) was included but this was less obvious for SPET without AC. The results suggest that OSEM is the preferable SPET reconstruction algorithm, producing more accurate rigidbody image registration when AC is used to remove artifacts due to non-uniform attenuation in the thoracic region. Registration performance deteriorated with decreasing planar projection count. The presence of the body boundary in the SPET image and matching fields of view were shown not to affect the registration performance substantially but pre-processing steps such as CT intensity windowing did improve registration accuracy. Non-rigid registration based on SCR was also investigated. The proposed algorithm for non-rigid registration is based on overlapping image blocks defined on a 3D grid pattern and a multi-level strategy. The transformation vector field, representing image deformation is found by translating each block so as to maximize the local similarity measure. The resulting sparsely sampled vector field is interpolated using a Gaussian function to ensure a locally smooth transformation. Comparisons were performed to test the effectiveness of SCR, MI and NMI in 3D intra- and inter-modality registration. The accuracy of the technique was evaluated on digital phantoms and on patient data. SCR demonstrated a better non-rigid registration than MI when sparse sampling was used for image block matching. For the high-resolution MR-MR image of brain region, the proposed algorithm was successful, placing 92% of image voxels within less than or equal to 2 voxels of the true position. Where one of the images had low resolution (e.g. in CT-SPET, MR-SPET registration), the accuracy and robustness deteriorated profoundly. In the current implementation, a 3D registration process takes about 10 minutes to complete on a stand alone Pentium IV PC with 1.7 GHz CPU and 256 Mbytes random access memory on board.

In AR systems, registration is one of the most difficult problems currently limiting their applications. In this paper, we proposed a simple registration method using projective reconstruction. This method consists of two steps: embedding and tracking. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In tracking, a projective reconstruction technique in computer vision is used to track the four specified points to compute the modelview transformation for augmentation. This method is simple as only four points need to be specified at the embedding stage, and the virtual object can then be easily augmented in a real video sequence. In addition, it can be extended to a common scenario using a common projective matrix. The proposed method has three advantages: (1) It is fast because the linear least square method can be used to estimate the related matrix in the algorithm and it is not necessary to calculate the fundamental matrix in the extended case; (2) A virtual object can still be superimposed on a related area even if some parts of the specified area are occluded during the augmentation process; and (3) This method is robust because it remains effective even when not all the reference points are detected during the augmentation process (in the rendering process), as long as at least six pairs of related reference point correspondences can be found. Several projective matrices obtained from the authors’ previous work, which are unrelated with the present AR system, were tested on this extended registration method. Experiments showed that these projective matrices can also be utilized for tracking the specified points. / Singapore-MIT Alliance (SMA)

An important part of stereo vision consists of finding and matching points in two images which correspond to the same physical element in the scene. We show that zeros of curvature of curves are perspective projection invariants and can therefore be used to find corresponding points. They can be used to help solve the registration problem (Longuet-Higgins, 1982) and to obtain the correct depth when a curve enters the forbidden zone (Krol and van de Grind, 1982). They are also relevant to theories for representing image curves. We consider the stability of these zeros of curvature.

We formulate and interpret several multi-modal registration methods in the context of a unified statistical and information theoretic framework. A unified interpretation clarifies the implicit assumptions of each method yielding a better understanding of their relative strengths and weaknesses. Additionally, we discuss a generative statistical model from which we derive a novel analysis tool, the "auto-information function", as a means of assessing and exploiting the common spatial dependencies inherent in multi-modal imagery. We analytically derive useful properties of the "auto-information" as well as verify them empirically on multi-modal imagery. Among the useful aspects of the "auto-information function" is that it can be computed from imaging modalities independently and it allows one to decompose the search space of registration problems.

We develop efficient techniques for the non-rigid registration of medical images by using representations that adapt to the anatomy found in such images. Images of anatomical structures typically have uniform intensity interiors and smooth boundaries. We create methods to represent such regions compactly using tetrahedra. Unlike voxel-based representations, tetrahedra can accurately describe the expected smooth surfaces of medical objects. Furthermore, the interior of such objects can be represented using a small number of tetrahedra. Rather than describing a medical object using tens of thousands of voxels, our representations generally contain only a few thousand elements. Tetrahedra facilitate the creation of efficient non-rigid registration algorithms based on finite element methods (FEM). We create a fast, FEM-based method to non-rigidly register segmented anatomical structures from two subjects. Using our compact tetrahedral representations, this method generally requires less than one minute of processing time on a desktop PC. We also create a novel method for the non-rigid registration of gray scale images. To facilitate a fast method, we create a tetrahedral representation of a displacement field that automatically adapts to both the anatomy in an image and to the displacement field. The resulting algorithm has a computational cost that is dominated by the number of nodes in the mesh (about 10,000), rather than the number of voxels in an image (nearly 10,000,000). For many non-rigid registration problems, we can find a transformation from one image to another in five minutes. This speed is important as it allows use of the algorithm during surgery. We apply our algorithms to find correlations between the shape of anatomical structures and the presence of schizophrenia. We show that a study based on our representations outperforms studies based on other representations. We also use the results of our non-rigid registration algorithm as the basis of a segmentation algorithm. That algorithm also outperforms other methods in our tests, producing smoother segmentations and more accurately reproducing manual segmentations.

Radiotherapy is a potential curative option for liver cancer; however, respiratory motion creates uncertainty in treatment delivery. Advances in imaging and registration techniques can provide information regarding changes in respiratory motion. Currently image registration is challenged by computation and manual intervention. A Navigator Channel (NC) technique was developed to overcome these limitations. A population motion model was generated to predict patient-specific motion, while a point motion detection technique was developed to calculate the patient-specific liver edge motion from images. An adaptation technique uses the relative difference between the population and patient calculated liver edge motion to determine the patient's liver volume motion. The NC technique was tested on patient 4D-CT images for initial validation to determine the accuracy. Accuracy was less than 0.10 mm in liver edge detection and approximately 0.25 cm in predicting patient-specific motion. This technique can be used to ensure accurate treatment delivery for liver radiotherapy.