No description available.

No description available.

Bridge inspection and infrastructure inspection are critical steps in the lifecycle of the built environment. Emerging technologies and data are driving factors which are disrupting the traditional processes for conducting these inspections. Because inspections are mainly conducted visually by human inspectors, this paper focuses on improving the visual inspection process with data-driven approaches. Data driven approaches, however, require significant data, which was sparse in the existing literature. Therefore, this research first examined the present state of the existing data in the research domain. We reviewed hundreds of image-based visual inspection papers which used machine learning to augment the inspection process and from this, we compiled a comprehensive catalog of over forty available datasets in the literature and identified promising, emerging techniques and trends in the field. Based on our findings in our review we contributed six significant datasets to target gaps in data in the field. The six datasets comprised of structural material segmentation, corrosion condition state segmentation, crack detection, structural detail detection, and bearing condition state classification. The contributed datasets used novel annotation guidelines and benefitted from a novel semi-automated annotation process for both object detection and pixel-level detection models. Using the data obtained from our collected sources, task-appropriate deep learning models were trained. From these datasets and models, we developed a change detection algorithm to monitor damage evolution between two inspection videos and trained a GAN-Inversion model which generated hyper-realistic synthetic bridge inspection image data and could forecast a future deterioration state of an existing bridge element. While the application of machine learning techniques in civil engineering is not wide-spread yet, this research provides impactful contribution which demonstrates the advantages that data driven sciences can provide to more economically and efficiently inspect structures, catalog deterioration, and forecast potential outcomes. / Doctor of Philosophy / Bridge inspection and infrastructure inspection are critical steps in the lifecycle of the built environment. Emerging technologies and data are driving factors which are disrupting the traditional processes for conducting these inspections. Because inspections are mainly conducted visually by human inspectors, this paper focuses on improving the visual inspection process with data-driven approaches. Data driven approaches, however, require significant data, which was sparse in the existing literature. Therefore, this research first examined the present state of the existing data in the research domain. We reviewed hundreds of image-based visual inspection papers which used machine learning to augment the inspection process and from this, we compiled a comprehensive catalog of over forty available datasets in the literature and identified promising, emerging techniques and trends in the field. Based on our findings in our review we contributed six significant datasets to target gaps in data in the field. The six datasets comprised of structural material detection, corrosion condition state identification, crack detection, structural detail detection, and bearing condition state classification. The contributed datasets used novel labeling guidelines and benefitted from a novel semi-automated labeling process for the artificial intelligence models. Using the data obtained from our collected sources, task-appropriate artificial intelligence models were trained. From these datasets and models, we developed a change detection algorithm to monitor damage evolution between two inspection videos and trained a generative model which generated hyper-realistic synthetic bridge inspection image data and could forecast a future deterioration state of an existing bridge element. While the application of machine learning techniques in civil engineering is not widespread yet, this research provides impactful contribution which demonstrates the advantages that data driven sciences can provide to more economically and efficiently inspect structures, catalog deterioration, and forecast potential outcomes.

This thesis addresses three problems related to image registration, prediction and tracking, applied to Angiography and Oncology. For image analysis, various probabilistic models have been employed to characterize the image deformations, target motions and state estimations. (i) In Digital Subtraction Angiography (DSA), having a high quality visualization of the blood motion in the vessels is essential both in diagnostic and interventional applications. In order to reduce the inherent movement artifacts in DSA, non-rigid image registration is used before subtracting the mask from the contrast image. DSA image registration is a challenging problem, as it requires non-rigid matching across spatially non-uniform control points, at high speed. We model the problem of sub-pixel matching, as a labeling problem on a non-uniform Markov Random Field (MRF). We use quad-trees in a novel way to generate the non uniform grid structure and optimize the registration cost using graph-cuts technique. The MRF formulation produces a smooth displacement field which results in better artifact reduction than with the conventional approach of independently registering the control points. The above approach is further improved using two models. First, we introduce the concept of pivotal and non-pivotal control points. `Pivotal control points' are nodes in the Markov network that are close to the edges in the mask image, while 'non-pivotal control points' are identified in soft tissue regions. This model leads to a novel MRF framework and energy formulation. Next, we propose a Gaussian MRF model and solve the energy minimization problem for sub-pixel DSA registration using Random Walker (RW). An incremental registration approach is developed using quad-tree based MRF structure and RW, wherein the density of control points is hierarchically increased at each level M depending of the features to be used and the required accuracy. A novel numbering scheme of the control points allows us to reuse the computations done at level M in M + 1. Both the models result in an accelerated performance without compromising on the artifact reduction. We have also provided a CUDA based design of the algorithm, and shown performance acceleration on a GPU. We have tested the approach using 25 clinical data sets, and have presented the results of quantitative analysis and clinical assessment. (ii) In External Beam Radiation Therapy (EBRT), in order to monitor the intra fraction motion of thoracic and abdominal tumors, the lung diaphragm apex can be used as an internal marker. However, tracking the position of the apex from image based observations is a challenging problem, as it undergoes both position and shape variation. We propose a novel approach for tracking the ipsilateral hemidiaphragm apex (IHDA) position on CBCT projection images. We model the diaphragm state as a spatiotemporal MRF, and obtain the trace of the apex by solving an energy minimization problem through graph-cuts. We have tested the approach using 15 clinical data sets and found that this approach outperforms the conventional full search method in terms of accuracy. We have provided a GPU based heterogeneous implementation of the algorithm using CUDA to increase the viability of the approach for clinical use. (iii) In an adaptive radiotherapy system, irrespective of the methods used for target observations there is an inherent latency in the beam control as they involve mechanical movement and processing delays. Hence predicting the target position during `beam on target' is essential to increase the control precision. We propose a novel prediction model (called o set sine model) for the breathing pattern. We use IHDA positions (from CBCT images) as measurements and an Unscented Kalman Filter (UKF) for state estimation. The results based on 15 clinical datasets show that, o set sine model outperforms the state of the art LCM model in terms of prediction accuracy.

Indiana University-Purdue University Indianapolis (IUPUI) / We present a novel, variational and statistical approach for model-based segmentation. Our model generalizes the Chan-Vese model, proposed for concurrent segmentation of multiple objects embedded in the same image domain. We also propose a novel shape descriptor, namely the Multi-Compartment Distance Functions or mcdf. Our proposed framework for segmentation is two-fold: first, several training samples distributed across various classes are registered onto a common frame of reference; then, we use a variational method similar to Active Shape Models (or ASMs) to generate an average shape model and hence use the latter to partition new images. The key advantages of such a framework is: (i) landmark-free automated shape training; (ii) strict shape constrained model to fit test data. Our model can naturally deal with shapes of arbitrary dimension and topology(closed/open curves). We term our model Active Geometric Model, since it focuses on segmentation of geometric shapes. We demonstrate the power of the proposed framework in two important medical applications: one for morphology estimation of 3D Motor Neuron compartments, another for thickness estimation of Henle's Fiber Layer in the retina. We also compare the qualitative and quantitative performance of our method with that of several other state-of-the-art segmentation methods.

This thesis proposes a Visual Scene Change Detector(VSCD) system which is a system which involves four parts, image retrieval, image registration, image change detection and panorama creation. Two prestudies are conducted in order to find a proposed image registration method and a image retrieval method. The two found methods are then combined with a proposed image registration method and a proposed panorama creation method to form the proposed VSCD. The image retrieval prestudy evaluates a SIFT related method with a bag of words related method and finds the SIFT related method to be the superior method. The image change detection prestudy evaluates 8 different image change detection methods. Result from the image change detection prestudy shows that the methods performance is dependent on the image category and an ensemble method is the least dependent on the category of images. An ensemble method is found to be the best performing method followed by a range filter method and then a Convolutional Neural Network (CNN) method. Using a combination of the 2 image retrieval methods and the 8 image change detection method 16 different VSCD are formed and tested. The final result show that the VSCD comprised of the best methods from the prestudies is the best performing method. / Detta exjobb föreslår ett Visual Scene Change Detector(VSCD) system vilket är ett system som har 4 delar, image retrieval, image registration, image change detection och panorama creation. Två förstudier görs för att hitta en föreslagen image registration metod och en föreslagen panorama creation metod. De två föreslagna delarna kombineras med en föreslagen image registration och en föreslagen panorama creation metod för att utgöra det föreslagna VSCD systemet. Image retrieval förstudien evaluerar en ScaleInvariant Feature Transform (SIFT) relaterad method med en Bag of Words (BoW) relaterad metod och hittar att den SIFT relaterade methoden är bäst. Image change detection förstudie visar att metodernas prestanda är beroende av catagorin av bilder och att en enemble metod är minst beroende av categorin av bilder. Enemble metoden är hittad att vara den bästa presterande metoden följt av en range filter metod och sedan av en CNN metod. Genom att använda de 2 image retrieval metoder kombinerat med de 8 image change detection metoder är 16 st VSCD system skapade och testade. Sista resultatet visar att den VSCD som använder de bästa metoderna från förstudien är den bäst presterande VSCD.

Image registration methods underpin many analysis techniques in neuroimaging. They are essential in group studies when images of different individuals or different modalities need to be brought into a common reference frame. This thesis explores the potential of brain connectivity- driven alignment and develops surface registration techniques for magnetic resonance imaging (MRI), which is a noninvasive neuroimaging tool for probing function and structure of the human brain. The first part of this work develops a novel surface registration framework, based on free mesh deformations, which aligns cortical and subcortical surfaces by matching structural connectivity patterns derived using probabilistic tractography (diffusion-weighted MRI). Structural, i.e. white matter, connectivity is a good predictor of functional specialisation and structural connectivity-driven registration can therefore be expected to enhance the alignment of functionally homologous areas across subjects. The second part validates developed methods for cortical surfaces. Resting State Networks are used in an innovative way to delineate several functionally distinct regions, which were then used to quantify connectivity-driven registration performance by measuring the inter- subject overlap before and after registration. Consequently, the proposed method is assessed using an independent imaging modality and the results are compared to results from state-of-the-art cortical geometry-driven surface registration methods. A connectivity-driven registration pipeline is also developed for, and applied to, the surfaces of subcortical structures such as the thalamus. It is carefully validated on a set of artificial test examples and compared to another novel surface registration paradigm based on spherical wavelets. The proposed registration pipeline is then used to explore the differences in the alignment of two groups of subjects, healthy controls and Alzheimer's disease patients, to a common template. Finally, we propose how functional connectivity can be used instead of structural connectivity for driving registrations, as well as how the surface-based framework can be extended to a volumetric one. Apart from providing the benefits such as the improved functional alignment, we hope that the research conducted in this thesis will also represent the basis for the development of templates of structural and functional brain connectivity.

Non-rigid image registration is an important tool for analysing morphometric differences in subjects with Alzheimer's disease from structural magnetic resonance images of the brain. This thesis describes a novel probabilistic approach to non-rigid registration of medical images, and explores the benefits of its use in this area of neuroimaging. Many image registration approaches have been developed for neuroimaging. The vast majority suffer from two limitations: Firstly, the trade-off between image fidelity and regularisation requires selection. Secondly, only a point-estimate of the mapping between images is inferred, overlooking the presence of uncertainty in the estimation. This thesis introduces a novel probabilistic non-rigid registration model and inference scheme. This framework allows the inference of the parameters that control the level of regularisation, and data fidelity in a data-driven fashion. To allow greater flexibility, this model is extended to allow the level of data fidelity to vary across space. A benefit of this approach, is that the registration can adapt to anatomical variability and other image acquisition differences. A further advantage of the proposed registration framework is that it provides an estimate of the distribution of probable transformations. Additional novel contributions of this thesis include two proposals for exploiting the estimated registration uncertainty. The first of these estimates a local image smoothing filter, which is based on the registration uncertainty. The second approach incorporates the distribution of transformations into an ensemble learning scheme for statistical prediction. These techniques are integrated into standard frameworks for morphometric analysis, and are demonstrated to improve the ability to distinguish subjects with Alzheimer's disease from healthy controls.

Les techniques de description des éléments caractéristiques d’une image sont omniprésentes dans de nombreuses applications de vision par ordinateur. Nous proposons à travers ce manuscrit une extension, pour décrire (représenter) et apparier les éléments caractéristiques des images. L’extension proposée consiste en une approche originale pour apprendre, ou estimer, la présence sémantique des éléments caractéristiques locaux dans les images. L’information sémantique obtenue est ensuite exploitée, en conjonction avec le paradigme de sac-de-mots, pour construire un descripteur d’image performant. Le descripteur résultant, est la combinaison de deux types d’informations, locale et contextuelle-sémantique. L’approche proposée peut être généralisée et adaptée à n’importe quel descripteur local d’image, pour améliorer fortement ses performances spécialement quand l’image est soumise à des conditions d’imagerie contraintes. La performance de l’approche proposée est évaluée avec des images réelles aussi bien dans les deux domaines, 2D que 3D. Nous avons abordé dans le domaine 2D, un problème lié à l’appariement des éléments caractéristiques dans des images. Dans le domaine 3D, nous avons résolu les problèmes d’appariement et alignement des vues partielles tridimensionnelles. Les résultats obtenus ont montré qu’avec notre approche, les performances sont nettement meilleures par rapport aux autres méthodes existantes. / This manuscript presents an extension of feature description and matching strategies by proposing an original approach to learn the semantic information of local features. This semantic is then exploited, in conjunction with the bag-of-words paradigm, to build a powerful feature descriptor. The approach, ended up by combining local and context information into a single descriptor, is also a generalized method for improving the performance of the local features, in terms of distinctiveness and robustness under geometric image transformations and imaging conditions. The performance of the proposed approach is evaluated on real world data sets as well as in both the 2D and 3D domains. The 2D domain application addresses the problem of image feature matching while in 3D domain, we resolve the issue of matching and alignment of multiple range images. The evaluation results showed our approach performs significantly better than expected results as well as in comparison with other methods.

L'objectif de cette thèse est de faciliter le diagnostic du cancer de la vessie. Durant une cystoscopie, un endoscope est introduit dans la vessie pour explorer la paroi interne de l'organe qui est visualisée sur un écran. Cependant, le faible champ de vue de l'instrument complique le diagnostic et le suivi des lésions. Cette thèse présente des algorithmes pour la création de cartes bi- et tridimensionnelles à large champ de vue à partir de vidéo-séquences cystoscopiques. En utilisant les avancées récentes dans le domaine de la minimisation d'énergies discrètes, nous proposons des fonctions coût indépendantes des transformations géométriques requises pour recaler de façon robuste et précise des paires d'images avec un faible recouvrement spatial. Ces transformations sont requises pour construire des cartes lorsque des trajectoires d'images se croisent ou se superposent. Nos algorithmes détectent automatiquement de telles trajectoires et réalisent une correction globale de la position des images dans la carte. Finalement, un algorithme de minimisation d'énergie compense les faibles discontinuités de textures restantes et atténue les fortes variations d'illuminations de la scène. Ainsi, les cartes texturées sont uniquement construites avec les meilleures informations (couleurs et textures) pouvant être extraites des données redondantes des vidéo-séquences. Les algorithmes sont évalués quantitativement et qualitativement avec des fantômes réalistes et des données cliniques. Ces tests mettent en lumière la robustesse et la précision de nos algorithmes. La cohérence visuelle des cartes obtenues dépassent celles des méthodes de cartographie de la vessie de la littérature / The aim of this thesis is to facilitate bladder cancer diagnosis. The reference clinical examination is cystoscopy, where an endoscope, inserted into the bladder, allows to visually explore the organ's internal walls on a monitor. The main restriction is the small field of view (FOV) of the instrument, which complicates lesion diagnosis, follow-up and treatment traceability.In this thesis, we propose robust and accurate algorithms to create two- and three-dimensional large FOV maps from cystoscopic video-sequences. Based on recent advances in the field of discrete energy minimization, we propose transformation-invariant cost functions, which allow to robustly register image pairs, related by large viewpoint changes, with sub-pixel accuracy. The transformations linking such image pairs, which current state-of-the-art bladder image registration techniques are unable to robustly estimate, are required to construct maps with several overlapping image trajectories. We detect such overlapping trajectories automatically and perform non-linear global map correction. Finally, the proposed energy minimization based map compositing algorithm compensates small texture misalignments and attenuates strong exposure differences. The obtained textured maps are composed by a maximum of information/quality available from the redundant data of the video-sequence. We evaluate the proposed methods both quantitatively and qualitatively on realistic phantom and clinical data sets. The results demonstrate the robustness of the algorithms, and the obtained maps outperform state-of-the-art approaches in registration accuracy and global map coherence