In Finance, the use of Automated Trading Systems (ATS) on markets is growing every year and the trades generated by an algorithm now account for most of orders that arrive at stock exchanges (Kissell, 2020). Historically, these systems were based on advanced statistical methods and signal processing designed to extract trading signals from financial data. The recent success of Machine Learning has attracted the interest of the financial community. Reinforcement Learning is a subcategory of machine learning and has been broadly applied by investors and researchers in building trading systems (Kissell, 2020). In this thesis, we address the issue that deep reinforcement learning may be susceptible to sampling errors and over-fitting and propose a robust deep reinforcement learning method that integrates techniques from reinforcement learning and robust optimization. We back-test and compare the performance of the developed algorithm, Robust DDPG, with UBAH (Uniform Buy and Hold) benchmark and other RL algorithms and show that the robust algorithm of this research can reduce the downside risk of an investment strategy significantly and can ensure a safer path for the investor’s portfolio value.

This thesis engages on the differences between the numerical simulations of sheet metal drawing process performed using AutoForm software from AutoForm Engineering, Swiss company and actual results measured using non-contact measurement system Argus from German GOM company. This work used DC06 material (whose mechanical properties were determined at Technical University of Liberec) for comparison. The calculated results of simulations were compared to measured results of actual plate thickness.

Avec la révolution moléculaire en biologie, une compréhension des mécanismes de la relation génotype-phénotype est devenue possible. Récemment, les progrès réalisés dans la synthèse et le séquençage de l’ADN ont permis le développement d’expériences de deep-mutational scanning capable de quantifier divers phénotypes pour un ensemble de génotypes sur toute la longueur d’un gène. Ces ensembles de données sont non seulement intéressants en eux-mêmes, mais permettent également de tester de manière rigoureuse des modèles phénotypiques quantitatifs. Nous avons utilisé cette technologie pour caractériser les cartes séquence-fitness de 3 systèmes bactériens modèles: un régulateur global, la CRP, une enzyme de résistance aux antibiotiques, la β-lactamase, et une petite voie métabolique constituée des enzymes AraA et AraB. Ces systèmes ont été choisis pour éclairer les rôles de différentes caractéristiques dans la formation de la relation génotype-fitness (réseaux de régulations, stabilité des protéines et flux métabolique). Nous constatons que la tendance globale des effets sur le fitness semble prévaloir sur les tendances spécifiques. Ceci nous conduit à penser qu’une grande partie de la relation entre le génotype et le fitness pourrait être expliquée à partir de la forme des fonctions de phénotype-fitness. Par ailleurs, nous voyons que la caractérisation de la relation génotype-fitness dans différents systèmes peut être un moyen puissant d’obtenir des informations sur les phénotypes pertinents. / With the molecular revolution in Biology, a mechanistic understanding of the genotype-phenotype relationship became possible. Recently, advances in DNA synthesis and sequencing have enabled the development of deep-mutational scanning experiments, capable of scoring comprehensive libraries of genotypes for a variety of phenotypes over the length of entire genes. Such datasets are not only interesting in themselves, but also allow rigorous testing of quantitative phenotypic models. We used this technology to characterise sequence-fitness maps for 3 model bacterial systems: a global regulator, CRP, an antibiotic-resistance enzyme, β-lactamase, and a small metabolic pathway, consisting of the enzymes AraA and AraB. These different systems were chosen to illuminate the roles of different mechanistic features in shaping the genotype-fitness relationship (regulatory wiring, protein stability and metabolic flux). We find that smooth patterns of fitness effects tend to prevail over idiosyncrasy, indicating that much of the genotype-fitness relationship could be understood from the global shape of smooth underlying phenotype-fitness functions. On the flip side, we see that characterising the genotype-fitness relationship in different systems can be a powerful way to glean phenotypic insights.

Fluorescence microscopy is an indispensable tool for biology to study the spatio-temporal dynamics of cells, tissues, and developing organisms. Modern imaging modalities, such as light-sheet microscopy, are able to acquire large three- dimensional volumes with high spatio-temporal resolution for many hours or days, thereby routinely generating Terabytes of image data in a single experiment. The quality of these images, however, is limited by the optics of the microscope, the signal-to-noise ratio of acquisitions, the photo-toxic effects of illumination, and the distortion of light by the sample. Additionally, the serial operation mode of most microscopy experiments, where large data sets are first acquired and only afterwards inspected and analyzed, excludes the possibility to optimize image quality during acquisition by automatically adapting the microscope parameters. These limits make certain observations difficult or impossible, forcing trade-offs between imaging speed, spatial resolution, light exposure, and imaging depth. This thesis is concerned with addressing several of these challenges with computational methods. First, I present methods for visualizing and processing the volumetric data from a microscope in real-time, i.e. at the acquisition rate of typical experiments, which is a prerequisite for the development of adaptive microscopes. I propose a low-discrepancy sampling strategy that enables the seamless display of large data sets during acquisition, investigate real-time compatible denoising, convolution, and deconvolution methods, and introduce a low-rank decomposition strategy for common deblurring tasks. Secondly, I propose a computational tractable method to simulate the interaction of light with realistically large biological tissues by combining a GPU-accelerated beam propagation method with a novel multiplexing scheme. I demonstrate that this approach enables to rigorously simulate the wave-optical image formation in light-sheet microscopes, to numerically investigate correlative effects in scattering tissues, and to elucidate the optical properties of the inverted mouse retina. Finally, I propose a data-driven restoration approach for fluorescence microscopy images based on convolutional neural networks (Care) that leverages sample and imaging specific prior knowledge. By demonstrating the superiority of this approach when compared to classical methods on a variety of problems, ranging from restoration of high quality images from low signal-to-noise-ratio acquisitions, to projection of noisy developing surface, isotropic recovery from anisotropic volumes, and to the recovery of diffraction-limited structures from widefield images alone, I show that Care is a flexible and general method to solve fundamental restoration problems in fluorescence microscopy.

Marin energi har stor potential att på ett relativt miljövänligt sätt utvinna energi ur bl.a. vind, vågor och strömmar. Prototyper och kraftverk för att skörda energi ur tidvattenströmmar har de senaste 10 åren blivit mer populärt, inte minst för att uppnå de miljökraven som ställts internationellt. Minesto är ett företag som utvecklar ett tidvattenkraftverk som heter Deep Green, som har ett väldigt unikt utförande, och har analyserats och jämförts mot två andra relevanta konkurrerande tidvattenkraftverk, DeltaStream och Seagen S. Studien har fokuserats på de vanligaste utförandena och variation vad gäller transmission, fundament, installation, strategi för att utföra underhåll och reparationer, reglering och elnätsanslutningar, för att sedan på ett mer strukturerat sätt förklara och beskriva de tre kraftverken. Deep Green är en så kallad tidvattensdrake. Tidvattensdraken består av en vinge med gondol och turbin som är monterad i havsbotten med ett tjuder. När tidvattnet förs över vingen börjar Deep Green att färdas framåt, på grund av den lyftkraft som bildas över vingen, i en bana formad som en åtta. Kraftverket uppnår sin märkeffekt på 0,5MW vid tidvattenströmmar på 1,4 m/s. DeltaStream och Seagen S är båda tidvattenkraftverk med horisontal axiala monterade turbiner, dvs. samma princip som vindkraftverk men tillämpad under vatten. DeltaStream och Seagen S producerar vid märkeffekt 1,2MW respektive 1,2MW - 2,0MW vid strömhastighet på 3,1 m/s respektive 2,5 m/s. Den jämförande analysen påvisar att Deep Green har störst potential och var bäst på 8 av 18 punkter. Analysen sammanställdes och rangordnades genom poängen 1-3, med avseende på egenskaper i förhållande till varandra då kraftverket med bäst egenskap under en rad fick 3 poäng och den minst bra får 1 poäng. Saknas uppgift ges ett poäng och likadana/liknande egenskaper ger 2 eller 1 poäng beroende på egenskap. Denna sammanställning gav Deep Green 42 poäng, Seagen S 36 poäng och DeltaStream 34 poäng. / Marine Energy has a great potential to extract energy in a relatively environmentally stable order from e.g. wind, waves and streams. Prototypes and power plants to extract energy from tidal streams have gotten quite popular the last 10 years, none the less because of the international environmental agreements. Minesto is a business that’s developing a tidal power plant called Deep Green that has a very unique design, and has been analysed and compared with two other relevant competitive tidal power plants, DeltaStream and Seagen S. This study has focused on the most common designs and variation by transmission, foundation, installation, strategy for maintenance and repairs, control and grid connections, to in a more structured way explain and introduce the three tidal power plants. Deep Green is a so called tidal kite. The tidal kite consists of a wing with nacelle and a turbine, and the unit is mounted to the seabed with a tether. Deep Green starts to move forward when the tide flows over the wing, due to the lift force, in a 8 shaped trajectory. The power plant reaches its max power extraction of 0,5 MW in tides from 1,4 m/s. DeltaStream and Seagen S are both tidal power plants with horizontally mounted turbines, by the same principle as wind power plants but design for underwater use. DeltaStream and Seagen S are producing 1,2 MW and 1,2MW – 2,0MW in tides from 3,1 m/s and 2,5 m/s, respectively. The comparing analysis shows that Deep Green has the greatest potential and was the best in 8 out of 18 points The analysis was compiled and was ranked through the points 1-3, with respect to characteristics relative to each other where the power plant with the best characteristic in one row got 3 points and the least good characteristic got 1 point. Is any information missing is 1 point given and equivalent properties get 2 or 1 point depending on the property. This compilation gave Deep Green 42 points, Seagen S 36 points and DeltaStream 34 points.

Many emergency departments are today suffering from a overcrowding of people seeking care. The first stage in seeking care is being prioritised in different orders depending on symptoms by a doctor or nurse called medical triage. This is a cumbersome process that could be subject of automatisation. This master thesis investigates the possibility of using reinforcement learning for performing medical triage of patients. A deep Q-learning approach is taken for designing the agent for the environment together with the two extensions of using double Q-learning and a duelling network architecture. The agent is deployed to train in two different environments. The goal for the agent in the first environment is to ask questions to a patient and then decide, when enough information has been collected, how the patient should be prioritised. The second environment makes the agent decide which questions should be asked to the patient and then a separate classifier is used with the information gained to perform the actual triage decision of the patient. The training and testing process of the agent in the two environments reveal difficulties in exploring the environment efficiently and thoroughly. It was also shown that defining a reward function for the environments that guides the agent into asking valuable questions and learninga stopping condition for asking questions is a complicated task. Suitable future work is discussed that would, in combination with the work performed in this paper, create a better reinforcement learning model that could potentially show more promising results in the task of performing medical triage of patients.

Deep learning methods have the ability to accurately predict and interpret what data represents. However, the decision making of a deep learning model is not comprehensible for humans. This is a problem for sectors like market surveillance which needs clarity in the decision making of the used algorithms. This thesis aimed to investigate how a deep learning model can be constructed to make the decision making of the model humanly comprehensible, and to investigate the potential impact on classification performance. A literature study was performed and publicly available explanation methods were collected. The explanation methods LIME, SHAP, model distillation and SHAP TreeExplainer were implemented and evaluated on a ResNet trained on three different time-series datasets. A decision tree was used as the student model for model distillation, where it was trained with both soft and hard labels. A survey was conducted to evaluate if the explanation method could increase comprehensibility. The results were that all methods could improve comprehensibility for people with experience in machine learning. However, none of the methods could provide full comprehensibility and clarity of the decision making. The model distillation reduced the performance compared to the ResNet model and did not improve the performance of the student model. / Djupinlärningsmetoder har egenskapen att förutspå och tolka betydelsen av data. Däremot så är djupinlärningsmetoders beslut inte förståeliga för människor. Det är ett problem för sektorer som marknadsövervakning som behöver klarhet i beslutsprocessen för använda algoritmer. Målet för den här uppsatsen är att undersöka hur en djupinlärningsmodell kan bli konstruerad för att göra den begriplig för en människa, och att undersöka eventuella påverkan av klassificeringsprestandan. En litteraturstudie genomfördes och publikt tillgängliga förklaringsmetoder samlades. Förklaringsmetoderna LIME, SHAP, modelldestillering och SHAP TreeExplainer blev implementerade och utvärderade med en ResNet modell tränad med tre olika dataset. Ett beslutsträd användes som studentmodell för modelldestillering och den blev tränad på båda mjuka och hårda etiketter. En undersökning genomfördes för att utvärdera om förklaringsmodellerna kan förbättra förståelsen av modellens beslut. Resultatet var att alla metoder kan förbättra förståelsen för personer med förkunskaper inom maskininlärning. Däremot så kunde ingen av metoderna ge full förståelse och insyn på hur beslutsprocessen fungerade. Modelldestilleringen minskade prestandan jämfört med ResNet modellen och förbättrade inte prestandan för studentmodellen.

Le travail présenté dans cette thèse porte principalement sur le développement de méthodes permettant d'extraire des descripteurs pertinents des images acquises des articulateurs dans les chants rares : les polyphonies traditionnelles Corses, Sardes, la musique Byzantine, ainsi que le Human Beat Box. Nous avons collecté des données, et employons des méthodes d'apprentissage statistique pour les modéliser, notamment les méthodes récentes d'apprentissage profond (Deep Learning).Nous avons étudié dans un premier temps des séquences d'images échographiques de la langue apportant des informations sur l'articulation, mais peu lisibles sans connaissance spécialisée en échographie. Nous avons développé des méthodes pour extraire de façon automatique le contour supérieur de la langue montré par les images échographiques. Nos travaux ont donné des résultats d'extraction du contour de la langue comparables à ceux obtenus dans la littérature, ce qui pourrait permettre des applications en pédagogie du chant.Ensuite, nous avons prédit l'évolution des paramètres du filtre qu'est le conduit vocal depuis des séquences d'images de langue et de lèvres, sur des bases de données constituées de voyelles isolées puis de chants traditionnels Corses. L'utilisation des paramètres du filtre du conduit vocal, combinés avec le développement d'un modèle acoustique de source vocale exploitant l'enregistrement électroglottographique, permet de synthétiser des extraits de voix chantée en utilisant les images articulatoires (de la langue et des lèvres)et l'activité glottique, avec des résultats supérieurs à ceux obtenus avec les techniques existant dans la littérature. / This thesis reports newly developed methods which can be applied to extract relevant features from articulator images in rare singing: traditional Corsican and Sardinian polyphonies, Byzantine music, as well as Human Beat Box. We collected data, and modeled these using machine learning methods, specifically novel deep learning methods. We first modelled tongue ultrasound image sequences, carrying relevant articulatory information which would otherwise be difficult to interpret without specialized skills in ultrasound imaging. We developed methods to extract automatically the superior contour of the tongue displayed on ultrasound images. Our tongue contour extraction results are comparable with those obtained in the literature, which could lead to applications in singing pedagogy. Afterwards, we predicted the evolution of the vocal tract filter parameters from sequences of tongue and lip images, first on isolated vowel databases then on traditional Corsican singing. Applying the predicted filter parameters, combined with the development of a vocal source acoustic model exploiting electroglottographic recordings, allowed us to synthesize singing voice excerpts using articulatory images (of tongue and lips) and glottal activity, with results superior to those obtained using existing technics reported in the literature.

Object detection is one of the key tasks of environment perception for highly automated vehicles. To achieve a high level of performance and fault tolerance, automated vehicles are equipped with an array of different sensors to observe their environment. Perception systems for automated vehicles usually rely on Bayesian fusion methods to combine information from different sensors late in the perception pipeline in a highly abstract, low-dimensional representation. Newer research on deep learning object detection proposes fusion of information in higher-dimensional space directly in the convolutional neural networks to significantly increase performance. However, the resulting deep learning architectures violate key non-functional requirements of a real-world safety-critical perception system for a series-production vehicle, notably modularity, fault tolerance and traceability. This dissertation presents a modular multimodal perception architecture for detecting objects using camera, lidar and radar data that is entirely based on deep learning and that was designed to respect above requirements. The presented method is applicable to any region-based, two-stage object detection architecture (such as Faster R-CNN by Ren et al.). Information is fused in the high-dimensional feature space of a convolutional neural network. The feature map of a convolutional neural network is shown to be a suitable representation in which to fuse multimodal sensor data and to be a suitable interface to combine different parts of object detection networks in a modular fashion. The implementation centers around a novel neural network architecture that learns a transformation of feature maps from one sensor modality and input space to another and can thereby map feature representations into a common feature space. It is shown how transformed feature maps from different sensors can be fused in this common feature space to increase object detection performance by up to 10% compared to the unimodal baseline networks. Feature extraction front ends of the architecture are interchangeable and different sensor modalities can be integrated with little additional training effort. Variants of the presented method are able to predict object distance from monocular camera images and detect objects from radar data. Results are verified using a large labeled, multimodal automotive dataset created during the course of this dissertation. The processing pipeline and methodology for creating this dataset along with detailed statistics are presented as well.

Die Arbeit ist Teil eines aktuellen Forschungsprojekts, bei der ein dynamischer zweidimensionaler Verkehrsflusssimulator zur Beschreibung der Binnenschifffahrt auf einer ca. 220 km langen Strecke auf dem Niederrhein entwickelt werden soll. Ziel dieser Arbeit ist es, ein Schifffolgemodell mithilfe von Deep Learning Ansätzen umzusetzen und mittels geeigneter Beschleunigung ein kollisionsfreies Folgen zu realisieren. Dabei sind die gesetzlichen Randbedingungen (Verkehrsregeln, Mindestabstände) sowie hydrodynamische und physikalische Gesetzmäßigkeiten wie minimale und maximale Beschleunigungen und Geschwindigkeiten zu berücksichtigen. Nach der Analyse des Systems sowie der notwendigen Parameter, wird ein Modell entworfen und die Modellparameter bestimmt. Unter Berücksichtigung der Modellparameter wird ein Agent ausgewählt und das System in MATLAB implementiert. Die Parameter sind so gestaltet, dass sich damit ein allgemeines Folgemodell ergibt und beispielsweise auch ein Autofolgemodell realisieren lässt.:1 Einleitung 1.1 Ziel der Arbeit 1.2 Aufbau der Arbeit 2 Stand der Technik 2.1 Traditionelle Folgemodelle 2.2 Reinforcement Learning 2.2.1 Modell 2.2.2 State-value function 2.3 Deep Reinforcement Learning 2.3.1 Künstliches neuronales Netz 3 Mathematische Grundlagen 3.1 Künstliche Neuronen 3.1.1 Aktivierungsfunktionen 3.2 Normierung 3.3 Funktionstypen 4 Analyse 4.1 Analyse der Systemfunktionen der Software 5 Modell 5.1 Aufbau 5.2 Approximatoren 5.3 Parameter 5.4 Szenarien 6 Agent 6.1 Auswahl des Agenten 6.2 Twin-Delayed Deterministic Policy Gradient (TD3) 7 Implementierung 7.1 Environment 7.1.1 Rewardfunktion 7.2 Agent 7.2.1 Netzwerkarchitektur 7.2.1.1 Actor-Netzwerk 7.2.1.2 Critic-Netzwerk 7.2.1.3 Rauschprozesse 7.3 Hyperparameter 7.4 Sonstige Parameter 8 Trainingsprozess 45 8.1 Ornstein-Uhlenbeck-Prozess 8.2 Algorithmus 9 Validierung 9.1 Fahrverhalten bei verschiedenen Charakteristika 9.2 Vergleich mit dem Intelligent Driver Model 10 Zusammenfassung und Ausblick Literaturverzeichnis