The Advanced Scatterometer (ASCAT) is a satellite-based remote sensing instrument designed for measuring wind speed and direction over the Earth's oceans. This thesis aims to expand and improve the capabilities of ASCAT by adding rain detection and advancing wind retrieval. Additionally, this expansion to ASCAT serves as evidence of Artificial Intelligence (AI) techniques learning both novel and traditional methods in remote sensing. I apply semantic segmentation to ASCAT measurements to detect rain over the oceans, enhancing capabilities to monitor global precipitation. I use two common neural network architectures and train them on measurements from the Tropical Rainfall Measuring Mission (TRMM) collocated with ASCAT measurements. I apply the same semantic segmentation techniques on wind retrieval in order to create a machine learning model that acts as an inverse Geophysical Model Function (GMF). I use three common neural network architectures and train the models on ASCAT data collocated with European Centre for Medium-Range Weather Forecasts (ECMWF) wind vector data. I successfully increase the capabilities of the ASCAT satellite to detect rainfall in Earth's oceans, with the ability to retrieve wind vectors without a GMF or Maximum Likelihood Estimation (MLE).

Numerous of high-impact applications involve predictive modeling of real-world data. This spans from hospital readmission prediction for enhanced patient care up to event detection in power systems for grid stabilization. Developing performant machine learning models necessitates extensive high-quality training data, ample labeled samples, and training and testing datasets derived from identical distributions. Though, such methodologies may be impractical in applications where obtaining labeled data is expensive or challenging, the quality of data is low, or when challenged with covariate or concept shifts. Our emphasis was on devising transfer learning methods to address the inherent challenges across two distinct applications.We delved into a notably challenging transfer learning application that revolves around predicting hospital readmission risks using electronic health record (EHR) data to identify patients who may benefit from extra care. Readmission models based on EHR data can be compromised by quality variations due to manual data input methods. Utilizing high-quality EHR data from a different hospital system to enhance prediction on a target hospital using traditional approaches might bias the dataset if distributions of the source and target data are different. To address this, we introduce an Early Readmission Risk Temporal Deep Adaptation Network, ERR-TDAN, for cross-domain knowledge transfer. A model developed using target data from an urban academic hospital was enhanced by transferring knowledge from high-quality source data. Given the success of our method in learning from data sourced from multiple hospital systems with different distributions, we further addressed the challenge and infeasibility of developing hospital-specific readmission risk prediction models using data from individual hospital systems. Herein, based on an extension of the previous method, we introduce an Early Readmission Risk Domain Generalization Network, ERR-DGN. It is adept at generalizing across multiple EHR data sources and seamlessly adapting to previously unseen test domains. In another challenging application, we addressed event detection in electrical grids where dependencies are spatiotemporal, highly non-linear, and non-linear systems using high-volume field-recorded data from multiple Phasor Measurement Units (PMUs). Existing historical event logs created manually do not correlate well with the corresponding PMU measurements due to scarce and temporally imprecise labels. Extending event logs to a more complete set of labeled events is very costly and often infeasible to obtain. We focused on utilizing a transfer learning method tailored for event detection from PMU data to reduce the need for additional manual labeling. To demonstrate the feasibility, we tested our approach on large datasets collected from the Western and Eastern Interconnections of the U.S.A. by reusing a small number of carefully selected labeled PMU data from a power system to detect events from another. Experimental findings suggest that the proposed knowledge transfer methods for healthcare and power system applications have the potential to effectively address the identified challenges and limitations. Evaluation of the proposed readmission models show that readmission risk predictions can be enhanced when leveraging higher-quality EHR data from a different site, and when trained on data from multiple sites and subsequently applied to a novel hospital site. Moreover, labels scarcity in power systems can be addressed by a transfer learning method in conjunction with a semi-supervised algorithm that is capable of detecting events based on minimal labeled instances. / Computer and Information Science

Ces travaux de recherche visent à développer des méthodes d’apprentissage automatique pour l’analyse de l’électroencéphalogramme (EEG) continu. L’EEG continu est une modalité avantageuse pour l’évaluation fonctionnelle des états cérébraux en réanimation ou pour d’autres applications. Cependant son utilisation aujourd’hui demeure plus restreinte qu’elle ne pourrait l’être, car dans la plupart des cas l’interprétation est effectuée visuellement par des spécialistes.Les sous-parties de ce travail s’articulent autour de l’évaluation pronostique du coma post-anoxique, choisie comme application pilote. Un petit nombre d’enregistrement longue durée a été réalisé, et des enregistrements existants ont été récupérés au CHU Grenoble.Nous commençons par valider l’efficacité des réseaux de neurones profonds pour l’analyse EEG d’échantillons bruts. Nous choisissons à cet effet de travailler sur la classification de stades de sommeil. Nous utilisons un réseau de neurones convolutionnel adapté pour l’EEG que nous entrainons et évaluons sur le jeu de données SHHS (Sleep Heart Health Study). Cela constitue le premier system neuronal à cette échelle (5000 patients) pour l’analyse du sommeil. Les performances de classification atteignent ou dépassent l’état de l’art.En utilisation réelle, pour la plupart des applications cliniques le défi principal est le manque d’annotations adéquates sur les patterns EEG ou sur de court segments de données (et la difficulté d’en établir). Les annotations disponibles sont généralement haut niveau (par exemple, le devenir clinique) est sont donc peu nombreuses. Nous recherchons comment apprendre des représentations compactes de séquences EEG de façon non-supervisée/semi-supervisée. Le domaine de l’apprentissage non supervisé est encore jeune. Pour se comparer aux travaux existants nous commençons avec des données de type image, et investiguons l’utilisation de réseaux adversaires génératifs (GANs) pour l’apprentissage adversaire non-supervisé de représentations. La qualité et la stabilité de différentes variantes sont évaluées. Nous appliquons ensuite un GAN de Wasserstein avec pénalité sur les gradients à la génération de séquences EEG. Le système, entrainé sur des séquences mono-piste de patients en coma post anoxique, est capable de générer des séquences réalistes. Nous développons et discutons aussi des idées originales pour l’apprentissage de représentations en alignant des distributions dans l’espace de sortie du réseau représentatif.Pour finir, les signaux EEG multipistes ont des spécificités qu’il est souhaitable de prendre en compte dans les architectures de caractérisation. Chaque échantillon d’EEG est un mélange instantané des activités d’un certain nombre de sources. Partant de ce constat nous proposons un système d’analyse composé d’un sous-système d’analyse spatiale suivi d’un sous-système d’analyse temporelle. Le sous-système d’analyse spatiale est une extension de méthodes de séparation de sources construite à l’aide de couches neuronales avec des poids adaptatifs pour la recombinaison des pistes, c’est à dire que ces poids ne sont pas appris mais dépendent de caractéristiques du signal d’entrée. Nous montrons que cette architecture peut apprendre à réaliser une analyse en composantes indépendantes, si elle est entrainée sur une mesure de non-gaussianité. Pour l’analyse temporelle, des réseaux convolutionnels classiques utilisés séparément sur les pistes recombinées peuvent être utilisés. / The objective of this research is to explore and develop machine learning methods for the analysis of continuous electroencephalogram (EEG). Continuous EEG is an interesting modality for functional evaluation of cerebral state in the intensive care unit and beyond. Today its clinical use remains more limited that it could be because interpretation is still mostly performed visually by trained experts. In this work we develop automated analysis tools based on deep neural models.The subparts of this work hinge around post-anoxic coma prognostication, chosen as pilot application. A small number of long-duration records were performed and available existing data was gathered from CHU Grenoble. Different components of a semi-supervised architecture that addresses the application are imagined, developed, and validated on surrogate tasks.First, we validate the effectiveness of deep neural networks for EEG analysis from raw samples. For this we choose the supervised task of sleep stage classification from single-channel EEG. We use a convolutional neural network adapted for EEG and we train and evaluate the system on the SHHS (Sleep Heart Health Study) dataset. This constitutes the first neural sleep scoring system at this scale (5000 patients). Classification performance reaches or surpasses the state of the art.In real use for most clinical applications, the main challenge is the lack of (and difficulty of establishing) suitable annotations on patterns or short EEG segments. Available annotations are high-level (for example, clinical outcome) and therefore they are few. We search how to learn compact EEG representations in an unsupervised/semi-supervised manner. The field of unsupervised learning using deep neural networks is still young. To compare to existing work we start with image data and investigate the use of generative adversarial networks (GANs) for unsupervised adversarial representation learning. The quality and stability of different variants are evaluated. We then apply Gradient-penalized Wasserstein GANs on EEG sequences generation. The system is trained on single channel sequences from post-anoxic coma patients and is able to generate realistic synthetic sequences. We also explore and discuss original ideas for learning representations through matching distributions in the output space of representative networks.Finally, multichannel EEG signals have specificities that should be accounted for in characterization architectures. Each EEG sample is an instantaneous mixture of the activities of a number of sources. Based on this statement we propose an analysis system made of a spatial analysis subsystem followed by a temporal analysis subsystem. The spatial analysis subsystem is an extension of source separation methods built with a neural architecture with adaptive recombination weights, i.e. weights that are not learned but depend on features of the input. We show that this architecture learns to perform Independent Component Analysis if it is trained on a measure of non-gaussianity. For temporal analysis, standard (shared) convolutional neural networks applied on separate recomposed channels can be used.

Fluorescence microscopy is an indispensable tool for biology to study the spatio-temporal dynamics of cells, tissues, and developing organisms. Modern imaging modalities, such as light-sheet microscopy, are able to acquire large three- dimensional volumes with high spatio-temporal resolution for many hours or days, thereby routinely generating Terabytes of image data in a single experiment. The quality of these images, however, is limited by the optics of the microscope, the signal-to-noise ratio of acquisitions, the photo-toxic effects of illumination, and the distortion of light by the sample. Additionally, the serial operation mode of most microscopy experiments, where large data sets are first acquired and only afterwards inspected and analyzed, excludes the possibility to optimize image quality during acquisition by automatically adapting the microscope parameters. These limits make certain observations difficult or impossible, forcing trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. This thesis is concerned with addressing several of these challenges with computational methods. First, I present methods for visualizing and processing the volumetric data from a microscope in real-time, i.e. at the acquisition rate of typical experiments, which is a prerequisite for the development of adaptive microscopes. I propose a low-discrepancy sampling strategy that enables the seamless display of large data sets during acquisition, investigate real-time compatible denoising, convolution, and deconvolution methods, and introduce a low-rank decomposition strategy for common deblurring tasks. Secondly, I propose a computational tractable method to simulate the interaction of light with realistically large biological tissues by combining a GPU-accelerated beam propagation method with a novel multiplexing scheme. I demonstrate that this approach enables to rigorously simulate the wave-optical image formation in light-sheet microscopes, to numerically investigate correlative effects in scattering tissues, and to elucidate the optical properties of the inverted mouse retina. Finally, I propose a data-driven restoration approach for fluorescence microscopy images based on convolutional neural networks (Care) that leverages sample and imaging specific prior knowledge. By demonstrating the superiority of this approach when compared to classical methods on a variety of problems, ranging from restoration of high quality images from low signal-to-noise-ratio acquisitions, to projection of noisy developing surface, isotropic recovery from anisotropic volumes, and to the recovery of diffraction-limited structures from widefield images alone, I show that Care is a flexible and general method to solve fundamental restoration problems in fluorescence microscopy.

Deep learning methods have the ability to accurately predict and interpret what data represents. However, the decision making of a deep learning model is not comprehensible for humans. This is a problem for sectors like market surveillance which needs clarity in the decision making of the used algorithms. This thesis aimed to investigate how a deep learning model can be constructed to make the decision making of the model humanly comprehensible, and to investigate the potential impact on classification performance. A literature study was performed and publicly available explanation methods were collected. The explanation methods LIME, SHAP, model distillation and SHAP TreeExplainer were implemented and evaluated on a ResNet trained on three different time-series datasets. A decision tree was used as the student model for model distillation, where it was trained with both soft and hard labels. A survey was conducted to evaluate if the explanation method could increase comprehensibility. The results were that all methods could improve comprehensibility for people with experience in machine learning. However, none of the methods could provide full comprehensibility and clarity of the decision making. The model distillation reduced the performance compared to the ResNet model and did not improve the performance of the student model. / Djupinlärningsmetoder har egenskapen att förutspå och tolka betydelsen av data. Däremot så är djupinlärningsmetoders beslut inte förståeliga för människor. Det är ett problem för sektorer som marknadsövervakning som behöver klarhet i beslutsprocessen för använda algoritmer. Målet för den här uppsatsen är att undersöka hur en djupinlärningsmodell kan bli konstruerad för att göra den begriplig för en människa, och att undersöka eventuella påverkan av klassificeringsprestandan. En litteraturstudie genomfördes och publikt tillgängliga förklaringsmetoder samlades. Förklaringsmetoderna LIME, SHAP, modelldestillering och SHAP TreeExplainer blev implementerade och utvärderade med en ResNet modell tränad med tre olika dataset. Ett beslutsträd användes som studentmodell för modelldestillering och den blev tränad på båda mjuka och hårda etiketter. En undersökning genomfördes för att utvärdera om förklaringsmodellerna kan förbättra förståelsen av modellens beslut. Resultatet var att alla metoder kan förbättra förståelsen för personer med förkunskaper inom maskininlärning. Däremot så kunde ingen av metoderna ge full förståelse och insyn på hur beslutsprocessen fungerade. Modelldestilleringen minskade prestandan jämfört med ResNet modellen och förbättrade inte prestandan för studentmodellen.

Le travail présenté dans cette thèse porte principalement sur le développement de méthodes permettant d'extraire des descripteurs pertinents des images acquises des articulateurs dans les chants rares : les polyphonies traditionnelles Corses, Sardes, la musique Byzantine, ainsi que le Human Beat Box. Nous avons collecté des données, et employons des méthodes d'apprentissage statistique pour les modéliser, notamment les méthodes récentes d'apprentissage profond (Deep Learning).Nous avons étudié dans un premier temps des séquences d'images échographiques de la langue apportant des informations sur l'articulation, mais peu lisibles sans connaissance spécialisée en échographie. Nous avons développé des méthodes pour extraire de façon automatique le contour supérieur de la langue montré par les images échographiques. Nos travaux ont donné des résultats d'extraction du contour de la langue comparables à ceux obtenus dans la littérature, ce qui pourrait permettre des applications en pédagogie du chant.Ensuite, nous avons prédit l'évolution des paramètres du filtre qu'est le conduit vocal depuis des séquences d'images de langue et de lèvres, sur des bases de données constituées de voyelles isolées puis de chants traditionnels Corses. L'utilisation des paramètres du filtre du conduit vocal, combinés avec le développement d'un modèle acoustique de source vocale exploitant l'enregistrement électroglottographique, permet de synthétiser des extraits de voix chantée en utilisant les images articulatoires (de la langue et des lèvres)et l'activité glottique, avec des résultats supérieurs à ceux obtenus avec les techniques existant dans la littérature. / This thesis reports newly developed methods which can be applied to extract relevant features from articulator images in rare singing: traditional Corsican and Sardinian polyphonies, Byzantine music, as well as Human Beat Box. We collected data, and modeled these using machine learning methods, specifically novel deep learning methods. We first modelled tongue ultrasound image sequences, carrying relevant articulatory information which would otherwise be difficult to interpret without specialized skills in ultrasound imaging. We developed methods to extract automatically the superior contour of the tongue displayed on ultrasound images. Our tongue contour extraction results are comparable with those obtained in the literature, which could lead to applications in singing pedagogy. Afterwards, we predicted the evolution of the vocal tract filter parameters from sequences of tongue and lip images, first on isolated vowel databases then on traditional Corsican singing. Applying the predicted filter parameters, combined with the development of a vocal source acoustic model exploiting electroglottographic recordings, allowed us to synthesize singing voice excerpts using articulatory images (of tongue and lips) and glottal activity, with results superior to those obtained using existing technics reported in the literature.

Object detection is one of the key tasks of environment perception for highly automated vehicles. To achieve a high level of performance and fault tolerance, automated vehicles are equipped with an array of different sensors to observe their environment. Perception systems for automated vehicles usually rely on Bayesian fusion methods to combine information from different sensors late in the perception pipeline in a highly abstract, low-dimensional representation. Newer research on deep learning object detection proposes fusion of information in higher-dimensional space directly in the convolutional neural networks to significantly increase performance. However, the resulting deep learning architectures violate key non-functional requirements of a real-world safety-critical perception system for a series-production vehicle, notably modularity, fault tolerance and traceability. This dissertation presents a modular multimodal perception architecture for detecting objects using camera, lidar and radar data that is entirely based on deep learning and that was designed to respect above requirements. The presented method is applicable to any region-based, two-stage object detection architecture (such as Faster R-CNN by Ren et al.). Information is fused in the high-dimensional feature space of a convolutional neural network. The feature map of a convolutional neural network is shown to be a suitable representation in which to fuse multimodal sensor data and to be a suitable interface to combine different parts of object detection networks in a modular fashion. The implementation centers around a novel neural network architecture that learns a transformation of feature maps from one sensor modality and input space to another and can thereby map feature representations into a common feature space. It is shown how transformed feature maps from different sensors can be fused in this common feature space to increase object detection performance by up to 10% compared to the unimodal baseline networks. Feature extraction front ends of the architecture are interchangeable and different sensor modalities can be integrated with little additional training effort. Variants of the presented method are able to predict object distance from monocular camera images and detect objects from radar data. Results are verified using a large labeled, multimodal automotive dataset created during the course of this dissertation. The processing pipeline and methodology for creating this dataset along with detailed statistics are presented as well.

Die Arbeit ist Teil eines aktuellen Forschungsprojekts, bei der ein dynamischer zweidimensionaler Verkehrsflusssimulator zur Beschreibung der Binnenschifffahrt auf einer ca. 220 km langen Strecke auf dem Niederrhein entwickelt werden soll. Ziel dieser Arbeit ist es, ein Schifffolgemodell mithilfe von Deep Learning Ansätzen umzusetzen und mittels geeigneter Beschleunigung ein kollisionsfreies Folgen zu realisieren. Dabei sind die gesetzlichen Randbedingungen (Verkehrsregeln, Mindestabstände) sowie hydrodynamische und physikalische Gesetzmäßigkeiten wie minimale und maximale Beschleunigungen und Geschwindigkeiten zu berücksichtigen. Nach der Analyse des Systems sowie der notwendigen Parameter, wird ein Modell entworfen und die Modellparameter bestimmt. Unter Berücksichtigung der Modellparameter wird ein Agent ausgewählt und das System in MATLAB implementiert. Die Parameter sind so gestaltet, dass sich damit ein allgemeines Folgemodell ergibt und beispielsweise auch ein Autofolgemodell realisieren lässt.:1 Einleitung 1.1 Ziel der Arbeit 1.2 Aufbau der Arbeit 2 Stand der Technik 2.1 Traditionelle Folgemodelle 2.2 Reinforcement Learning 2.2.1 Modell 2.2.2 State-value function 2.3 Deep Reinforcement Learning 2.3.1 Künstliches neuronales Netz 3 Mathematische Grundlagen 3.1 Künstliche Neuronen 3.1.1 Aktivierungsfunktionen 3.2 Normierung 3.3 Funktionstypen 4 Analyse 4.1 Analyse der Systemfunktionen der Software 5 Modell 5.1 Aufbau 5.2 Approximatoren 5.3 Parameter 5.4 Szenarien 6 Agent 6.1 Auswahl des Agenten 6.2 Twin-Delayed Deterministic Policy Gradient (TD3) 7 Implementierung 7.1 Environment 7.1.1 Rewardfunktion 7.2 Agent 7.2.1 Netzwerkarchitektur 7.2.1.1 Actor-Netzwerk 7.2.1.2 Critic-Netzwerk 7.2.1.3 Rauschprozesse 7.3 Hyperparameter 7.4 Sonstige Parameter 8 Trainingsprozess 45 8.1 Ornstein-Uhlenbeck-Prozess 8.2 Algorithmus 9 Validierung 9.1 Fahrverhalten bei verschiedenen Charakteristika 9.2 Vergleich mit dem Intelligent Driver Model 10 Zusammenfassung und Ausblick Literaturverzeichnis

Because of the long acquisition time of Positron Emission Tomography scanners, the reconstructed images are blurred by motion. We hereby propose a novel motion-correction maximum-likelihood expectation-maximization algorithm integrating 3D movements between the different gates estimated by a neural network trained on synthetic data with contrast invariance. We show that, compared to the classic reconstruction method, this algorithm can increase the image quality on realistic synthetic 3D data of a human body, in particular, the contrast of small carcinogenic lung lesions. For the detection of lesions of one cm on four gates for medium and high noise levels, the studied algorithm gave an increase of 45 to 130% of the Pearson correlation coefficient in comparison with classic reconstruction methods without deformations. / På grund av den långa insamlingstiden för Positron Emission Tomography skannrar, blir de rekonstruerade bilderna suddiga av rörelse. Vi föreslår härmed en ny algoritm för maximal sannolikhet för rörelsekorrigering förväntningar-maximering som integrerar 3D-rörelser mellan de olika grindarna uppskattade av ett neuralt nätverk tränat på syntetisk data med kontrastinvarians. Vi visar att, jämfört med den klassiska rekonstruktionsmetoden, kan denna algoritm öka bildkvaliteten på realistiska syntetiska 3D-data från en människokropp, i synnerhet kontrasten av små cancerframkallande lungskador. För detektion av lesioner på en cm på fyra grindar för medelhöga och höga ljudnivåer gav den studerade algoritmen en ökning med 45 till 130% av Pearsons korrelationskoefficient i jämförelse med klassisk rekonstruktionsmetod utan deformationer.

This work examines how Deep Learning(DL) are integrated with a specific web application. It is performed by creating various artifacts that examine the integration of a specific web application with DL. This is done with regards to future expansion of functionality and the value it offers to the stakeholders. The insights that arise during the work are communicated to the stakeholders through weekly meetings throughout the process. The paper ends with a conclusion that is based on the insight’s that are gained during the work. The conclusion is that the best method is the combination of two of the artifacts. A REST service developed in the Python language that can determine if an image contains animals or not. This REST service I used in an external program that works towards the same object storage that the system does. The program reads images from the storage and tests whether they are empty or not with through the REST-service. Pictures that are classified as empty will be removed from the systems object storage.