The use of complementary non-invasive imaging modalities has been proposed to track disease progression, particularly cancer, while simultaneously evaluating therapeutic efficacy. A major obstacle is a limited ability to compare parameters obtained from different modalities, especially those from exogenous contrast agents or tracers. We hypothesize that combining Magnetic Resonance Imaging (MRI) and Ultrasound (US) will improve characterization of the tumour microenvironment. In this study, we describe a co-registration apparatus that facilitates the acquisition of a priori co-registered MR and US images in vivo. This apparatus was validated using phantom data and it was found that the US slices can be selected to an accuracy of +/- 100µm translationally and +/- 2 degrees rotationally. Additionally, it was shown that MRI and US may provide complimentary information about the tumour microenvironment, but more work needs to be done to assess repeatability of dynamic contrast enhanced MRI and US.

Image registration is the process of aligning two images such that their mutual features overlap. This is of great importance in several medical applications. In 2008 a novel method for simultaneous T1, T2 and proton density quantification was suggested. The method is in the field of quantitative Magnetic Resonance Imaging or qMRI. In qMRI parameters are quantified by a pixel-to-pixel fit of the image intensity as a function of different MR scanner settings. The quantification depends on several volumes of different intensities to be aligned. If a patient moves during the data aquisition the datasets will not be aligned and the results are degraded due to this. Since the quantification takes several minutes there is a considerable risk of patient movements. In this master thesis three image registration methods are presented and a comparison in robustness and speed was made. The phase based algorithm was suited for this problem and limited to finding rigid motion. The other two registration algorithms, originating from the Statistical Parametrical Mapping, SPM, package, were used as references. The result shows that the pixel-to-pixel fit is greatly improved in the datasets with found motion. In the comparison between the different methods the phase based algorithm turned out to be both the fastest and the most robust method.

Image registration is widely used in different areas nowadays. Usually, the efficiency, accuracy, and robustness in the registration process are concerned in applications. This thesis studies these issues by presenting an efficient intensity-based mono-modality rigid 2D-3D image registration method and constructing a novel mathematical model for intensity-based multi-modality rigid image registration. For mono-modality image registration, an algorithm is developed using RapidMind Multi-core Development Platform (RapidMind) to exploit the highly parallel multi-core architecture of graphics processing units (GPUs). A parallel ray casting algorithm is used to generate the digitally reconstructed radiographs (DRRs) to efficiently reduce the complexity of DRR construction. The optimization problem in the registration process is solved by the Gauss-Newton method. To fully exploit the multi-core parallelism, almost the entire registration process is implemented in parallel by RapidMind on GPUs. The implementation of the major computation steps is discussed. Numerical results are presented to demonstrate the efficiency of the new method. For multi-modality image registration, a new model for computing mutual information functions is devised in order to remove the artifacts in the functions and in turn smooth the functions so that optimization methods can converge to the optimal solutions accurately and efficiently. With the motivation originating from the objective to harmonize the discrepancy between the image presentation and the mutual information definition in previous models, the new model computes the mutual information function using both the continuous image function representation and the mutual information definition for continuous random variables. Its implementation and complexity are discussed and compared with other models. The mutual information computed using the new model appears quite smooth compared with the functions computed by others. Numerical experiments demonstrate the accuracy and efficiency of optimization methods in the case that the new model is used. Furthermore, the robustness of the new model is also verified.

As the capabilities of autonomous vehicles increase, their use in situations that are dangerous or dull for humans is becoming more popular. Autonomous systems are currently being used in several military and civilian domains, including search and rescue operations, disaster relief coordination, infrastructure inspection and surveillance missions. In order to perform high level mission autonomy tasks, a method is required for the vehicle to localize itself, as well as generate a map of the environment. Algorithms which allow the vehicle to concurrently localize and create a map of its surroundings are known as solutions to the Simultaneous Localization and Mapping (SLAM) problem. Certain high level tasks, such as drivability analysis and obstacle avoidance, benefit from the use of a dense map of the environment, and are typically generated with the use of point cloud data. The point cloud data is incorporated into SLAM algorithms with scan registration techniques, which determine the relative transformation between two sufficiently overlapping point clouds. The Normal Distributions Transform (NDT) algorithm is a promising method for scan registration, however many issues with the NDT approach exist, including a poor convergence basin, discontinuities in the NDT cost function, and unreliable pose estimation in sparse, outdoor environments. This thesis presents methods to overcome the shortcomings of the NDT algorithm, in both 2D and 3D scenarios. To improve the convergence basin of NDT for 2D scan registration, the Multi-Scale k-Means NDT (MSKM-NDT) algorithm is presented, which divides a 2D point cloud using k-means clustering and performs the scan registration optimization over multiple scales of clustering. The k-means clustering approach generates fewer Gaussian distributions when compared to the standard NDT algorithm, allowing for evaluation of the cost function across all Gaussian clusters. Cost evaluation across all the clusters guarantees that the optimization will converge, as it resolves the issue of discontinuities in the cost function found in the standard NDT algorithm. Experiments demonstrate that the MSKM-NDT approach can be used to register partially overlapping scans with large initial transformation error, and that the convergence basin of MSKM-NDT is superior to NDT for the same test data. As k-means clustering does not scale well to 3D, the Segmented Greedy Cluster NDT (SGC-NDT) method is proposed as an alternative approach to improve and guarantee convergence using 3D point clouds that contain points corresponding to the ground of the environment. The SGC-NDT algorithm segments the ground points using a Gaussian Process (GP) regression model and performs clustering of the non ground points using a greedy method. The greedy clustering extracts natural features in the environment and generates Gaussian clusters to be used within the NDT framework for scan registration. Segmentation of the ground plane and generation of the Gaussian distributions using natural features results in fewer Gaussian distributions when compared to the standard NDT algorithm. Similar to MSKM-NDT, the cost function can be evaluated across all the clusters in the scan, resulting in a smooth and continuous cost function that guarantees convergence of the optimization. Experiments demonstrate that the SGC-NDT algorithm results in scan registrations with higher accuracy and better convergence properties than other state-of-the-art methods for both urban and forested environments.

Medical imaging plays an increasingly important role in modern healthcare. In medical imaging, it is often relevant to relate different images to each other, something which can prove challenging, since there rarely exists a pre-defined mapping between the pixels in different images. Hence, there is a need to find such a mapping/transformation, a procedure known as image registration. Over the years, image registration has been proved useful in a number of clinical situations. Despite this, current use of image registration in clinical practice is rather limited, typically only used for image fusion. The limited use is, to a large extent, caused by excessive computation times, lack of established validation methods/metrics and a general skepticism toward the trustworthiness of the estimated transformations in deformable image registration. This thesis aims to overcome some of the issues limiting the use of image registration, by proposing a set of technical contributions and two clinical applications targeted at improved clinical efficiency. The contributions are made in the context of a generic framework for non-parametric image registration and using an image registration method known as the Morphon.  In image registration, regularization of the estimated transformation forms an integral part in controlling the registration process, and in this thesis, two regularizers are proposed and their applicability demonstrated. Although the regularizers are similar in that they rely on local structure analysis, they differ in regard to implementation, where one is implemented as applying a set of filter kernels, and where the other is implemented as solving a global optimization problem. Furthermore, it is proposed to use a set of quadrature filters with parallel scales when estimating the phase-difference, driving the registration. A proposal that brings both accuracy and robustness to the registration process, as shown on a set of challenging image sequences. Computational complexity, in general, is addressed by porting the employed Morphon algorithm to the GPU, by which a performance improvement of 38-44x is achieved, when compared to a single-threaded CPU implementation. The suggested clinical applications are based upon the concept paint on priors, which was formulated in conjunction with the initial presentation of the Morphon, and which denotes the notion of assigning a model a set of properties (local operators), guiding the registration process. In this thesis, this is taken one step further, in which properties of a model are assigned to the patient data after completed registration. Based upon this, an application using the concept of anatomical transfer functions is presented, in which different organs can be visualized with separate transfer functions. This has been implemented for both 2D slice visualization and 3D volume rendering. A second application is proposed, in which landmarks, relevant for determining various measures describing the anatomy, are transferred to the patient data. In particular, this is applied to idiopathic scoliosis and used to obtain various measures relevant for assessing spinal deformity. In addition, a data analysis scheme is proposed, useful for quantifying the linear dependence between the different measures used to describe spinal deformities.

Abstract   Background Semi-automated segmentation using deformable registration of atlases consisting of pre-segmented patient images can facilitate the tedious task of delineating structures and organs in patients subjected to radiotherapy planning. However, a generic atlas based on a single patient may not function well enough due to the anatomical variation between patients. Fusion of segmentation proposals from multiple atlases has the potential to provide a better segmentation due to a more complete representation of the anatomical variation. Purpose The main goal of the present study was to investigate potential operator timesavings from use of atlas-based segmentation compared to manual segmentation of patients with prostate cancer. It was also anticipated that, and evaluated if, the use of semi-automated segmentation workflows would reduce the operator dependent variations in delineation. Materials and Methods A commercial atlas-based segmentation software (VelocityAI from Nucletron AB) was used with several atlases of consistently, protocol based, delineated CT images to create multiple-atlas segmentation proposals through deformable registration. The atlas that was considered most representative was selected to construct single generic atlas segmentation proposals. For fusion of the multiple-atlas segmentations an in-house developed algorithm, which includes information of local registration success was used in a MATLAB-environment[1]. The algorithm used weighted distance map calculations where weights represent probabilities of improving the segmentation results. Based on results from Sjöberg and Ahnesjö the probabilities were estimated using the cross correlation image similarity measure evaluated over a region within a certain distance from the segmentation. 10 patients were included in the study. Each patient was delineated three times, (a) manually by the radiation oncologist, (b) with a generic single-atlas segmentation and (c) with a fusion of multiple-atlas segmentations. For the methods (b) and (c) the radiation oncologist corrected the proposed segmentations blindly without using the result from method (a) as reference. The total number of atlases used for case (c) was 15. The operator time spent by the radiation oncologist was recorded separately for each method. In addition a grading was used to score how helpful the segmentation proposals were for the delineations. The Dice Similarity Coefficient, the Hausdorff distance and the segmented volumes were used to evaluate the similarity between the delineated structures and organs. Results An average time reduction of 26% was found when the radiation oncologist corrected the multiple atlas-based segmentation proposals as compared to manual segmentations. Due to more accurate segmentations and more time saved, segmentation with fused multiple-atlases (c) was superior to the generic single-atlas (b) method, which showed a time reduction of 17%. Hints of an affected intra- and inter-operator variability were seen. Conclusions Atlas-based segmentation saves time for the radiation oncologist but the segmentation proposals always need editing to be approved for dose planning. The atlases, the fusion of these and the software implementation needs to be improved for optimal results and to extend the clinically usefulness.

Background Semi-automated segmentation using deformable registration of atlases consisting of pre-segmented patient images can facilitate the tedious task of delineating structures and organs in patients subjected to radiotherapy planning. However, a generic atlas based on a single patient may not function well enough due to the anatomical variation between patients. Fusion of segmentation proposals from multiple atlases has the potential to provide a better segmentation due to a more complete representation of the anatomical variation. Purpose The main goal of the study was to investigate potential operator timesaving from editing of atlas-based segmentation compared to manual segmentation for head & neck cancer. Materials and Methods A commercial atlas-based segmentation software (VelocityAI from Nucletron AB) was used together with several expert generated and protocol-based atlases of delineated CT images to create multiple atlas segmentations through deformable registration. The atlas that was considered most universal was selected to construct single atlas segmentation proposals. For fusion of the multiple atlas segmentations an in-house developed algorithm, including information of local registration success was used in a MATLAB-environment1. The algorithm uses weighted distance map calculations where weights represent probabilities of improving the segmentation results. Based on previous results1 the probabilities were estimated using the cross correlation image similarity measure evaluated over a region within a certain distance from the segmentation. Ten patients were incorporated in the study. Each patient was delineated three times, (a) manually by the radiation oncologist, (b) with a single atlas segmentation and (c) with a fusion of multiple atlas segmentations. For the methods (b) and (c) the radiation oncologist corrected the proposed segmentations blindly without using the result from method (a) as reference. For case (c) a total number of 11 atlas segmentations were used. The time spent for segmenting or editing the segmentation proposals by the radiation oncologist was recorded separately for each method and each individual ROI. In addition a grading was used to score how helpful the candidate segmentation proposals were for the structure delineations. The Dice Similarity Coefficient, the Hausdorff distance and the volume were used to evaluate the similarity between the delineated structures. Results The results show a time reduction in the order of 40% when the radiation oncologist only has to correct the multiple atlas-based segmentation proposal compared to manual segmentation. When using single atlas the corresponding figure is 21%. Conclusions Using atlas-based segmentation can reduce the time needed for delineation in the head and neck area of patients admitted for radiotherapy. 1C. Sjöberg and A. Ahnesjö, Evaluation of atlas-based segmentation using probabilistic weighted distance maps, Manuscript, Uppsala University, 2011 / Bakgrund Atlasbaserad, semiautomatisk segmentering skulle kunna användas för att underlätta den för onkologen tidskrävande uppgiften med att manuellt segmentera strukturer och organ i patienter vid behandlingsplanering inför strålbehandling. Tidigare segmenterade atlaspatienter ger med hjälp av deformeringsalgoritmer segmenteringsförslag för strukturer i den aktuella patienten. Dessa kan sedan kontrolleras och editeras av onkologen med en tidsbesparing gentemot manuell segmentering som följd. En atlas som baserats på en enstaka individ (singelatlas) kan dock ha begränsningar när det gäller att täcka de anatomiska variationer som finns mellan olika patienter. Därför har metoder med fusionering av multipla segmenteringsförslag från en databas bestående av ett antal sedan tidigare segmenterade patienter (fusionerad multipelatlas) potential att ge ett bättre segmenteringsresultat. Syfte Huvudsyftet med arbetet var att undersöka de möjliga tidsbesparingar för onkologen som kan åstadkommas när editering av atlasbaserad segmentering används vid planering inför strålbehandling i huvud- och halsområdet istället för manuell segmentering Material och metoder En kommersiell, atlasbaserad segmenteringsprogramvara (VelocityAI från Nucletron AB) användes i studien. Genom att låta en erfaren onkolog segmentera ett antal CT-studier (11 st) enligt ett vedertaget protokoll skapades en databas av atlaser som sedan, via deformerbara registreringar, kunde generera lika många segmenteringsförslag för en nytillkommen patient. Den enskilda atlas som ansågs mest representativ valdes till att framställa segmenteringsförslaget för metoden med singelatlas. Till metoden med fusionerade multipla atlaser användes en lokalt utvecklad MATLAB-algoritm baserad på viktade    distansmappar. Vikterna representerar sannolikheten för förbättrat segmenteringsresultat och baseras på tidigare resultat1 där sannolikheterna bestämts utifrån en beräkning av likheterna mellan bilderna i ett visst område från den specifika segmenteringen. Tio patienter har inkluderats i studien. Varje patient segmenterades tre gånger, (a) manuellt, (b) med singelatlas och (c) med fusionerade multipla atlaser. För metoderna (b) och (c) editerades sedan segmenteringsförslagen av onkologen utan att denne fick använda resultatet från metod (a) som referens. För fallet med fusionerade multipla atlaser, (c), användes databasen med 11 atlaser. Tiden onkologen behövde för segmentering respektive editering av segmenteringsförslaget uppmättes i varje enskilt fall för jämförelse. Onkologen fick även göra en bedömning av hur hjälpsamt segmenteringsförslaget var i samband med editeringen. För utvärdering av resultaten användes Dice’s similaritetskoefficient, Hausdorff’s distansmått samt strukturernas volym. Resultat Resultaten visar på att en tidsbesparing i storleksordningen 40 % är rimlig när onkologen editerar förslag från fusioneringen av multipla atlassegmenteringar i jämförelse med manuell segmentering. Vid användning av singelatlas är motsvarande siffra 21 %. Slutsatser Användandet av atlasbaserad segmentering kan reducera tidsåtgången för segmentering av patienter inför strålbehandling i huvud-halsområdet. 1C. Sjöberg and A. Ahnesjö, Evaluation of atlas-based segmentation using probabilistic weighted distance maps, Manuscript, Uppsala University, 2011

Modern medicine has become reliant on medical imaging. Multiple modalities, e. g. magnetic resonance imaging (MRI), computed tomography (CT), etc. , are used to provide as much information about the patient as possible. The problem of geometrically aligning the resulting images is called image registration. Mutual information, an information theoretic similarity measure, allows for automated intermodal image registration algorithms. <br /><br /> In applications such as cancer therapy, diagnosticians are more concerned with the alignment of images over a region of interest such as a cancerous lesion, than over an entire image set. Attempts to register only the regions of interest, defined manually by diagnosticians, fail due to inaccurate mutual information estimation over the region of overlap of these small regions. <br /><br /> This thesis examines the region of union as an alternative to the region of overlap. We demonstrate that the region of union improves the accuracy and reliability of mutual information estimation over small regions. <br /><br /> We also present two new mutual information based similarity measures which allow for localized image registration by combining local and global image information. The new similarity measures are based on convex combinations of the information contained in the regions of interest and the information contained in the global images. <br /><br /> Preliminary results indicate that the proposed similarity measures are capable of localizing image registration. Experiments using medical images from computer tomography and positron emission tomography demonstrate the initial success of these measures. <br /><br /> Finally, in other applications, auto-detection of regions of interest may prove useful and would allow for fully automated localized image registration. We examine methods to automatically detect potential regions of interest based on local activity level and present some encouraging results.

Image registration is widely used in different areas nowadays. Usually, the efficiency, accuracy, and robustness in the registration process are concerned in applications. This thesis studies these issues by presenting an efficient intensity-based mono-modality rigid 2D-3D image registration method and constructing a novel mathematical model for intensity-based multi-modality rigid image registration. For mono-modality image registration, an algorithm is developed using RapidMind Multi-core Development Platform (RapidMind) to exploit the highly parallel multi-core architecture of graphics processing units (GPUs). A parallel ray casting algorithm is used to generate the digitally reconstructed radiographs (DRRs) to efficiently reduce the complexity of DRR construction. The optimization problem in the registration process is solved by the Gauss-Newton method. To fully exploit the multi-core parallelism, almost the entire registration process is implemented in parallel by RapidMind on GPUs. The implementation of the major computation steps is discussed. Numerical results are presented to demonstrate the efficiency of the new method. For multi-modality image registration, a new model for computing mutual information functions is devised in order to remove the artifacts in the functions and in turn smooth the functions so that optimization methods can converge to the optimal solutions accurately and efficiently. With the motivation originating from the objective to harmonize the discrepancy between the image presentation and the mutual information definition in previous models, the new model computes the mutual information function using both the continuous image function representation and the mutual information definition for continuous random variables. Its implementation and complexity are discussed and compared with other models. The mutual information computed using the new model appears quite smooth compared with the functions computed by others. Numerical experiments demonstrate the accuracy and efficiency of optimization methods in the case that the new model is used. Furthermore, the robustness of the new model is also verified.

An interesting problem in computer vision is that of image registration, which plays an important role in many vision-based recognition and motion analysis applications. Of particular interest among data registration problems are multimodal image registration problems, where the image data sets are acquired using different imaging modalities. There are several important issues that make real-world multimodal registration a difficult problem to solve. First, images are often characterized by illumination and contrast non-uniformities. Such image non-uniformities result in local minima along the convergence plane that make it difficult for local optimization schemes to converge to the correct solution. Second, real-world images are often contaminated with signal noise, making the extraction of meaningful features for comparison purposes difficult to accomplish. Third, feature space differences make performing direct comparisons between the different data sets with a reasonable level of accuracy a challenging problem. Finally, solving the multimodal registration problem can be computationally expensive for large images. This thesis presents a probabilistic complex phase representation (PCPR) objective function for registering images acquired using different imaging modalities. A probabilistic multi-scale approach is introduced to create image representations based on local phase relationships extracted using complex wavelets. An objective function is introduced for assessing the alignment between the images based on a Geman-McClure error distribution model between the probabilistic complex phase representations of the images. Experimental results show that the proposed PCPR objective function can provide improved registration accuracies when compared to existing objective functions.