L’objectif de ces travaux de recherche est d’étudier les méthodes d’évaluation de laqualité des images biométriques multimodales sur des échantillons acquis de manièrenon contrainte. De nombreuses s études ont noté l’importance de la qualité del’échantillon pour un système de reconnaissance ou un algorithme de comparaison,puisque la performance du système biométrique est intrinsèquement dépendant dela qualité des images de l’échantillon. Dès lors, la nécessité d’évaluer la qualitédes échantillons biométriques pour plusieurs modalités (empreintes digitales, iris,visage, etc.) est devenue primordiale notamment avec l’apparition de systèmesbiométriques multimodaux de haute précision.Après une introduction présentant un historique de la biométrie et des préceptesliés à la qualité des échantillons biométriques, nous présentons le concept d’évaluationde la qualité des échantillons pour plusieurs modalités. Les normes de qualitéISO / CEI récemment établies pour les empreintes digitales, l’iris et le visage sontprésentées. De plus, des approches d’évaluation de la qualité des échantillons conçuesspécifiquement pour les empreintes digitales avec et sans contact, pour l’iris(dont une image est capturée en proche infrarouge et dans le domaine visible),ainsi que le visage sont étudiées. Finalement, des techniques d’évaluation des performancesdes mesures de qualité des échantillons biométriques sont égalementétudiées.Sur la base des conclusions formulées suite à l’étude des solutions algorithmiques portant sur l’évaluation de la qualité des échantillons biométriques, nous proposonsun cadre commun pour l’évaluation de la qualité d’image biométrique pourplusieurs modalité. Après avoir étudié les attributs de qualité basés sur l’image parmodalité biométrique, nous examinons quelle intersection existe pour l’ensembledes modalités. Ensuite, nous sélectionnons et redéfinissons les attributs de qualitébasés sur l’image qui sont les plus importants afin de définir un cadre commun.Afin de relier ces attributs de qualité aux vrais échantillons biométriques,nous développons une nouvelle base de données de qualité d’image biométriquemulti-modalité qui contient des images échantillons de haute qualité et des imagesdégradées pour l’empreinte digitale acquise sans contact, l’iris (dont l’acquisitionest réalisée dans le spectre visible) et le visage. Les types de dégradation appliquéssont liés aux attributs de qualité qui sont communs aux diverses modalitéset qui sont basés sur l’image. Un autre aspect important du cadre commun proposéest la qualité de l’image et ses applications en biométrie. Nous avons d’abordintroduit et classifié les métriques de qualité d’image existantes, puis effectué unbref aperçu des métriques de qualité d’image sans référence, qui peuvent être appliquéespour l’évaluation de la qualité des échantillons biométriques. De plus, nousétudions comment les mesures de qualité d’image sans référence ont été utiliséespour l’évaluation de la qualité des empreintes digitales, de l’iris et des modalitésbiométriques du visage.Des expériences pour l’évaluation de la performance des métriques de qualitéd’image sans référence sur les images de visage et de l’iris sont effectuées. Lesrésultats expérimentaux indiquent qu’il existe plusieurs métriques qui peuventévaluer la qualité des échantillons biométriques de l’iris et du visage avec un fortcoefficient de correlation. La méthode obtenant les meilleurs résultats en termede performance est ré-entrainée sur des images d’empreintes digitales, ce qui permetd’augmenter significativement les performances du système de reconnaissancebiométrique.À travers le travail réalisé dans cette thèse, nous avons démontré l’applicabilitédes métriques de qualité d’image sans référence pour l’évaluation d’échantillonsbiométriques multi-modalité non contraints. / The aim of this research is to investigate multi-modality biometric image qualityassessment methods for unconstrained samples. Studies of biometrics noted thesignificance of sample quality for a recognition system or a comparison algorithmbecause the performance of the biometric system depends mainly on the qualityof the sample images. The need to assess the quality of multi-modality biometricsamples is increased with the requirement of a high accuracy multi-modalitybiometric systems.Following an introduction and background in biometrics and biometric samplequality, we introduce the concept of biometric sample quality assessment for multiplemodalities. Recently established ISO/IEC quality standards for fingerprint,iris, and face are presented. In addition, sample quality assessment approacheswhich are designed specific for contact-based and contactless fingerprint, nearinfrared-based iris and visible wavelength iris, as well as face are surveyed. Followingthe survey, approaches for the performance evaluation of biometric samplequality assessment methods are also investigated.Based on the knowledge gathered from the biometric sample quality assessmentchallenges, we propose a common framework for the assessment of multi-modalitybiometric image quality. We review the previous classification of image-basedquality attributes for a single biometric modality and investigate what are the commonimage-based attributes for multi-modality. Then we select and re-define themost important image-based quality attributes for the common framework. In order to link these quality attributes to the real biometric samples, we develop anew multi-modality biometric image quality database which has both high qualitysample images and degraded images for contactless fingerprint, visible wavelengthiris, and face modalities. The degradation types are based on the selected commonimage-based quality attributes. Another important aspect in the proposed commonframework is the image quality metrics and their applications in biometrics. Wefirst introduce and classify the existing image quality metrics and then conducteda brief survey of no-reference image quality metrics, which can be applied to biometricsample quality assessment. Plus, we investigate how no-reference imagequality metrics have been used for the quality assessment for fingerprint, iris, andface biometric modalities.The experiments for the performance evaluation of no-reference image qualitymetrics for visible wavelength face and iris modalities are conducted. The experimentalresults indicate that there are several no-reference image quality metricsthat can assess the quality of both iris and face biometric samples. Lastly, we optimizethe best metric by re-training it. The re-trained image quality metric canprovide better recognition performance than the original. Through the work carriedout in this thesis we have shown the applicability of no-reference image qualitymetrics for the assessment of unconstrained multi-modality biometric samples.

Multi-modality fusion is an area of research that has shown promising results in the domain of 2D and 3D object detection. However, multi-modality fusion methods have largely not been utilized in the domain of instance segmentation. This master’s thesis investigated if multi-modality fusion methods can be applied to deep learning instance segmentation models to improve their performance on multi-modality data. The two multi-modality fusion methods presented, input extension and feature fusions, were applied to a two-stage instance segmentation model, Mask R-CNN, and a single-stage instance segmentation model, RTMDet. Models were trained on different variations of preprocessed RGBD and ToF data provided by SICK IVP, as well as RGBD data from the publicly available NYUDepth dataset. The master’s thesis concludes that the multi-modality fusion method presented as feature fusion can be applied to the Mask R-CNN model to improve the networks performance by 1.8%points (1.8%pt.) bounding box mAP and 1.6%pt. segmentation mAP on SICK RGBD, 7.7%pt. bounding box mAP and 7.4%pt. segmentation mAP on ToF, and 7.4%pt. bounding box mAP and 7.4%pt. segmentation mAP on NYUDepth. The RTMDet model saw little to no improvements from the inclusion of depth but had similar baseline performance as the improved Mask R-CNN model that utilized feature fusion. The input extension method saw no improvements to performance as it faced technical implementation limitations.

Hyperpolarized 3Helium (HP 3He) magnetic resonance imaging (MRI) has provided considerable insights into the anatomical structures and localized physiological phenomenon involved in pulmonary ventilation. The increasing mortality rates of pulmonary diseases such as COPD, gives rise to the need for sensitive and regional assessments of early disease conditions in attempts to decrease mortality and improve lifestyles. Evaluation of the HP 3He MRI diffusion weighted measurements of lung microstructure, demonstrated a statistically significant relationship between microstructure expansion and degree of lung inflation at the time of imaging. The ability of HP 3He MRI to assess regional ventilation was validated against air volume change estimates of ventilation attainable via conventional MDCT in a cohort of 8 normal never smokers. Great correlations and slope were observed between the functional estimates, with similar gravitationally dependent-nondependent gradients throughout. A small but significant preferential helium distribution was observed in the nondependent regions, most likely due to gas density differences between air and helium. Further validation of HP 3He MRI's ability to assess regional ventilation, was carried via quantitative and qualitative assessments against xenon-enhanced MDCT (normal = 4, COPD = 2). The MRI based estimates were found to be insensitive to slow and fast ventilating regions, while superior in exhibiting ventilation defects. Similar gravitationally dependent - nondependent gradients were observed throughout, along with a homogenous distribution of the exogenous contrast agents. Coefficient of variation (COV) values followed similar trends in the normal subjects, while only one COPD subject demonstrated an increase from the normal population baseline. Acquisition differences including single vs. multi-breath and z-axis coverage could attribute to the quantitative differences observed. Evaluation of the density dependent distribution patterns of helium in a normal airway model via dynamic HP 3He MRI and computational fluid dynamics, demonstrated an increased preferential distribution in the nondependent airways, in agreement with the ventilation discrepancies previously observed. In combination with the developmental aspects of the presented research, we have validated the ability of HP 3He MRI to assess regional ventilation, via multiple quantitative assessments against conventional based and exogenously enhanced MDCT techniques and extracted the lung inflation level dependencies. Complimented with dynamic imaging and CFD simulations of helium distribution, these results provide insight into future considerations critical to the establishment of the technique as a surrogate to the ionizing radiation based modalities.

The use of complementary non-invasive imaging modalities has been proposed to track disease progression, particularly cancer, while simultaneously evaluating therapeutic efficacy. A major obstacle is a limited ability to compare parameters obtained from different modalities, especially those from exogenous contrast agents or tracers. We hypothesize that combining Magnetic Resonance Imaging (MRI) and Ultrasound (US) will improve characterization of the tumour microenvironment. In this study, we describe a co-registration apparatus that facilitates the acquisition of a priori co-registered MR and US images in vivo. This apparatus was validated using phantom data and it was found that the US slices can be selected to an accuracy of +/- 100µm translationally and +/- 2 degrees rotationally. Additionally, it was shown that MRI and US may provide complimentary information about the tumour microenvironment, but more work needs to be done to assess repeatability of dynamic contrast enhanced MRI and US.

The use of complementary non-invasive imaging modalities has been proposed to track disease progression, particularly cancer, while simultaneously evaluating therapeutic efficacy. A major obstacle is a limited ability to compare parameters obtained from different modalities, especially those from exogenous contrast agents or tracers. We hypothesize that combining Magnetic Resonance Imaging (MRI) and Ultrasound (US) will improve characterization of the tumour microenvironment. In this study, we describe a co-registration apparatus that facilitates the acquisition of a priori co-registered MR and US images in vivo. This apparatus was validated using phantom data and it was found that the US slices can be selected to an accuracy of +/- 100µm translationally and +/- 2 degrees rotationally. Additionally, it was shown that MRI and US may provide complimentary information about the tumour microenvironment, but more work needs to be done to assess repeatability of dynamic contrast enhanced MRI and US.

abstract: Hypoxia is a pathophysiological condition which results from lack of oxygen supply in tumors. The assessment of tumor hypoxia and its response to therapies can provide guidelines for optimization and personalization of therapeutic protocols for better treatment. Previous research has shown the difficulty in measuring hypoxia anatomically due to its heterogenous nature. This makes the study of hypoxia through various imaging modalities and mapping techniques crucial. The potential of hypoxia targeting T1 contrast agent GdDO3NI in generating hypoxia maps has been studied earlier. In this work, the similarities between hypoxia maps generated by MRI using GdDO3NI and pimonidazole based immunohistochemistry (IHC) in non-small cell lung carcinoma bearing mice have been studied. Six NCI-H1975 tumor-bearing mice were studied. All animal studies were approved by Arizona State University’s Institute of Animal Care and Use Committee (IACUC). Post co-injection of GdDO3NI and pimonidazole, T1 weighted 3D gradient echo MR images were acquired. For ex-vivo analysis of hypoxia, 30 μm thick tumor sections were obtained for each harvested tumor and were stained for pimonidazole and counter-stained with DAPI for nuclear staining. Pimonidazole (PIMO) is clinically used as a “gold standard” hypoxia marker. The key process involved stacking and iterative registration based on quality metric SSIM (Structural Similarity) Index of DAPI stained images of 5 consecutive tumor sections to produce a 3D volume stack of 150 μm thickness. Information from the 3D volume is combined to produce one final slide by averaging. The same registration transform was applied to stack the pimonidazole images which were previously thresholded to highlight hypoxic regions. The registered IHC stack was then co-registered with a single thresholded T1 weighted gradient echo MRI slice of the same location (~156 μm thick) using an elastic B-splines transform. The same transform was applied to achieve the co-registration of pimonidazole and MR percentage enhancement image. Image similarity index after the co-registration was found to be greater than 0.5 for 5 of the animals suggesting good correlation. R2 values were calculated for both hypoxic regions as well as tumor boundaries. All the tumors showed a high boundary correlation value of R2 greater than 0.8. Half of the animals showed high R2 values greater than 0.5 for hypoxic fractions. The RMSE values for the co-registration of all the animals were found to be low further suggesting better correspondence and validating the MR based hypoxia imaging. / Dissertation/Thesis / Masters Thesis Biomedical Engineering 2018

As demand for wind power continues to grow, it is becoming increasingly important to minimize the risk, characterized by the variance, that is associated with long-term power forecasts. This thesis investigated variance reduction in power forecasts from wind farm layout optimization.The problem was formulated as a multi-objective optimization one of maximizing mean-plant-power and minimizing variance. The ε−constraint method was used to solve the bi-objectiveproblem in a two-step optimization framework where two sequential optimizations are performed. The first is maximizing mean wind farm power alone and the second, minimizing variance with a constraint on the mean power which is the value from the first optimization. The results show that the variance in power estimates can be reduced by up to 30%, without sacrificing mean-plant-power for the different farm sizes and wind conditions studied. This reduction is attributed to the multi-modality of the design space which allows for unique solutions of high mean plant power at different power variances. Thus, wind farms can be designed to maximize power capture with greater confidence.

Validating claims from misinformation is a highly challenging task that involves understanding how each factual assertion within the claim relates to a set of trusted source materials. Existing approaches often make coarse-grained predictions but fail to identify the specific aspects of the claim that are troublesome and the specific evidence relied upon. In this paper, we introduce a method and new benchmark for this challenging task. Our method predicts the fine-grained logical relationship of each aspect of the claim from a set of multimodal documents, which include text, image(s), video(s), and audio(s). We also introduce a new benchmark (M^3DC) of claims requiring multimodal multidocument reasoning, which we construct using a novel claim synthesis technique. Experiments show that our approach significantly outperforms state-of-the-art baselines on this challenging task on two benchmarks while providing finer-grained predictions, explanations, and evidence. / Master of Science / In today's world, we are constantly bombarded with information from various sources, making it difficult to distinguish between what is true and what is false. Validating claims and determining their truthfulness is an essential task that helps us separate facts from fiction, but it can be a time-consuming and challenging process. Current methods often fail to pinpoint the specific parts of a claim that are problematic and the evidence used to support or refute them. In this study, we present a new method and benchmark for fact-checking claims using multiple types of information sources, including text, images, videos, and audio. Our approach analyzes each aspect of a claim and predicts how it logically relates to the available evidence from these diverse sources. This allows us to provide more detailed and accurate assessments of the claim's validity. We also introduce a new benchmark dataset called M^3DC, which consists of claims that require reasoning across multiple sources and types of information. To create this dataset, we developed a novel technique for synthesizing claims that mimic real-world scenarios. Our experiments show that our method significantly outperforms existing state-of-the-art approaches on two benchmarks while providing more fine-grained predictions, explanations, and evidence. This research contributes to the ongoing effort to combat misinformation and fake news by providing a more comprehensive and effective approach to fact-checking claims.

Intersection crashes account for over one-third of all crashes in the U.S., and 39% of these result in injury or death. As part of a larger effort to develop and evaluate in-vehicle countermeasures to reduce the number of intersection-related crashes, haptic warnings and a combined haptic/auditory warning were explored and compared to combined visual/auditory warnings. The first phase of this study determined which haptic brake pulse warning candidate most often resulted in the driver successfully stopping for an intersection. Five brake pulse warnings were tested (varied with respect to jerk, duration, and the number of pulses). Participants receiving the haptic warnings were 38 times more likely to stop at the intersection than those receiving no warning and 7.6 times more likely to stop than those receiving a combined visual/auditory tone warning. The 600ms-3 pulses condition was advanced to the second phase because it provided the longest warning and had a more favorable subjective rating; it was then combined with an auditory verbal warning (urgent "STOP"). This phase determined whether the added verbal warning resulted in differences from the haptic warning alone. Although the warning was activated 7.62 m (25 ft) closer to the intersection in the second phase than in the first phase, there were no significant differences for the reaction times and distance to stop bar. Participants receiving the haptic plus auditory verbal warning were also 1.5 times more likely to stop than those who received the haptic warning alone. Overall, this study shows that haptic warnings show promise for warning drivers of impending intersection violations. Guidelines for haptic intersection warnings were developed, including a recommendation that haptic warnings be combined with auditory verbal warnings for increased warning effectiveness. / Master of Science

Medical imaging has seen a rapid development in its clinical use in assessment of treatment outcome, disease monitoring and diagnosis over the last few decades. Yet, the vast amount of available image data limits the practical use of this potentially very valuable source of information for radiologists and physicians. Therefore, the design of computer-aided medical image analysis is of great importance to imaging in clinical practice. This thesis deals with the problem of deformable image registration in the context of lung imaging, and addresses three of the major challenges involved in this challenging application, namely: designing an image similarity for multi-modal scans or scans of locally changing contrast, modelling of complex lung motion, which includes sliding motion, and approximately globally optimal mathematical optimisation to deal with large motion of small anatomical features. The two most important contributions made in this thesis are: the formulation of a multi-dimensional structural image representation, which is independent of modality, robust to intensity distortions and very discriminative for different image features, and a discrete optimisation framework, based on an image-adaptive graph structure, which enables a very efficient optimisation of large dense displacement spaces and deals well with sliding motion. The derived methods are applied to two different clinical applications in pulmonary image analysis: motion correction for breathing-cycle computed tomography (CT) volumes, and deformable multi-modal fusion of CT and magnetic resonance imaging chest scans. The experimental validation demonstrates improved registration accuracy, a high quality of the estimated deformations, and much lower computational complexity, all compared to several state-of-the-art deformable registration techniques.