The Contracts Register is a public database containing contracts concluded by public institutions. Due to the number of documents in the database, data analysis is proble- matic. The objective of this thesis is to find a machine learning approach for sorting the contracts into categories by their area of interest (real estate services, construction, etc.) and implement the approach for usage on the web portal Hlídač státu. A large number of categories and a lack of a tagged dataset of contracts complicate the solution. 1

We refactor the traditional two-step approach of automatic speech recognition for spoken language translation. Instead of conventional graphemes, we use phonemes as an intermediate speech representation. Starting with the acoustic model, we revise the cross-lingual transfer and propose a coarse-to-fine method providing further speed-up and performance boost. Further, we review the translation model. We experiment with source and target encoding, boosting the robustness by utilizing the fine-tuning and transfer across ASR and SLT. We empirically document that this conventional setup with an alternative representation not only performs well on standard test sets but also provides robust transcripts and translations on challenging (e.g., non-native) test sets. Notably, our ASR system outperforms commercial ASR systems. 1

Computational ECG (electrocardiogram) analysis enables accurate and faster diagnosis and early prediction of heart failure related symptoms (heart failure worsening). Machine learning, particularly deep learning, has been applied for ECG data successfully. The previous applications, however, either mainly focused on classifying occurrent, known patterns of on-going heart failure or heart failure related diseases such arrhythmia, which have undesirable predictability beforehand, or emphasizing on data from pre-processed public database data. In this dissertation, we developed an approach, however, does not fully capitalize on the potential of deep learning, which directly learns important features from raw input data without relying on a priori knowledge. Here, we present a deep transfer learning pipeline which combines an image-based pretrained deep neural network model with manifold learning to predict the precursors of heart failure (heart failure-worsening and recurrent heart failure related re-hospitalization) using raw ECG time series from wearable devices. In this dissertation, we used the unprocessed real-life ECG data from the SENTINEL-HF study by Dovancescu, et al. to predict the precursors of heart failure worsening. To extract rich features from ECG time series, we took a deep transfer learning approach where 1D time-series of five heartbeats were transformed to 2D images by Gramian Angular Summation Field (GASF) and then the pretrained models, VGG19 were used for feature extraction. Then, we applied UMAP (Uniform Manifold Approximation and Projection) to capture the manifold of the standardized feature space and reduce the dimension, followed by SVM (Support Vector Machine) training. Using our pipeline, we demonstrated that our classifier was able to predict heart failure worsening with 92.1% accuracy, 92.9% precision, 92.6% recall and F1 score of 0.93 bypassing the detection of known abnormal ECG patterns. In conclusion, we demonstrate the feasibility of early alerts of heart failure by predicting the precursor of heart failure worsening based on raw ECG signals. We expected that our approached provided an innovative method to assess the recovery and successfulness for the treatment patient received during the first hospitalization, to predict whether recurrent heart failure is likely to occur, and to evaluate whether the patient should be discharged.

While methods like learning-without-forgetting [11] and elastic weight consolidation [22] accomplish high-quality transfer learning while mitigating catastrophic forgetting, progressive techniques such as Deepmind’s progressive neural network accomplish this while completely nullifying forgetting. However, progressive systems like this strictly require task labels during test time. In this paper, I introduce a novel task recognizer built from anomaly detection autoencoders that is capable of detecting the nature of the required task from input data.Alongside a progressive neural network or other progressive learning system, this task-aware network is capable of operating without task labels during run time while maintaining any catastrophic forgetting reduction measures implemented by the task model.

Training autonomous agents that are capable of performing their assigned job without fail is the ultimate goal of deep reinforcement learning. This thesis introduces a dueling Quantile Regression Deep Q-network, where the network learns the state value quantile function and advantage quantile function separately. With this network architecture the agent is able to learn to control simulated robots in the Gazebo simulator. Carefully crafted reward functions and state spaces must be designed for the agent to learn in complex non-stationary environments. When trained for only 100,000 timesteps, the agent is able reach asymptotic performance in environments with moving and stationary obstacles using only the data from the inertial measurement unit, LIDAR, and positional information. Through the use of transfer learning, the agents are also capable of formation control and flocking patterns. The performance of agents with frozen networks is improved through advice giving in Deep Q-networks by use of normalized Q-values and majority voting.

Energy efficiency of district heating systems is of great interest to energy stakeholders. However, it is not uncommon that district heating systems fail to achieve the expected performance due to inappropriate operations. Night setback is one control strategy, which has been proved to be not a suitable setting for well-insulated modern buildings in terms of both economic and energy efficiency. Therefore, identification of a night setback control is vital to district heating companies to smoothly manage their heat energy distribution to their customers. This study is motivated to automate this identification process. The method used in this thesis is a Convolutional Neural Network(CNN) approach using the concept of transfer learning. 133 substations in Oslo are used in this case study to design a machine learning model that can identify a substation as night setback or non-night setback series. The results show that the proposed method can classify the substations with approximately 97% accuracy and 91% F1-score. This shows that the proposed method has a high potential to be deployed and used in practice to identify a night setback control in district heating substations.

A big hurdle for many companies to start using machine learning is that trending techniques need a huge amount of structured data. One potential way to reduce the need for data is taking advantage of previous knowledge from a related task. This is so called transfer learning. A basic description of it would be when you take a model trained on existing data and reuse that for another problem. The purpose of this master thesis is to investigate if transfer learning can reduce the need for data when faced with a new machine learning task which is, in particular, to use transfer learning on ultrasound spectrograms of weld joints for predictive maintenance. The base for transfer learning is VGGish, a convolutional neural network model trained on audio samples collected from YouTube videos. The pre-trained weights are kept, and the prediction layer is replaced with a new prediction layer consisting of two neurons. The whole model is re-trained on the ultrasound spectrograms. The dataset is restricted to a minimum of ten and a maximum of 100 training samples. The results are evaluated and compared to a regular convolutional neural network trained on the same data. The results show that transfer learning improves the test accuracy compared to the regular convolutional neural network when the dataset is small. This thesis project concludes that transfer learning can reduce the need for data when faced with a new machine learning task. The results indicate that transfer learning could be useful in the industry.

Background. Domain adaptation is described as, a model learning from a source data distribution and performing well on the target data. This concept, Domain adaptation is applied to assembly-line production tasks to perform an automatic quality inspection. Objectives. The aim of this master thesis is to apply this concept of 3D domain adaptation from synthetic images to real images. It is an attempt to bridge the gap between different domains (synthetic and real point cloud images), by implementing deep learning models that learn from synthetic 3D point cloud (CAD model images) and perform well on the actual 3D point cloud (3D Camera images). Methods. Through this course of thesis project, various methods for understand- ing the data and analyzing it for bridging the gap between CAD and CAM to make them similar is looked into. Literature review and controlled experiment are research methodologies followed during implementation. In this project, we experiment with four different deep learning models with data generated and compare their performance to know which deep learning model performs best for the data. Results. The results are explained through metrics i.e, accuracy and train time, which were the outcomes of each of the deep learning models after the experiment. These metrics are illustrated in the form of graphs for comparative analysis between the models on which the data is trained and tested on. PointDAN showed better results with higher accuracy compared to the other 3 models. Conclusions. The results attained show that domain adaptation for synthetic images to real images is possible with the data generated. PointDAN deep learning model which focuses on local feature alignment and global feature alignment with single-view point data shows better results with our data.

The process of energy consumption and monitoring of a buildingis time-consuming. Therefore, an feasible approach for using trans-fer learning is presented to decrease the necessary time to extract re-quired large dataset. The technique applies a bidirectional long shortterm memory recurrent neural network using sequence to sequenceprediction. The idea involves a training phase that extracts informa-tion and patterns of a building that is presented with a reasonablysized dataset. The validation phase uses a dataset that is not sufficientin size. This dataset was acquired through a related paper, the resultscan therefore be validated accordingly. The conducted experimentsinclude four cases that involve different strategies in training and val-idation phases and percentages of fine-tuning. Our proposed modelgenerated better scores in terms of prediction performance comparedto the related paper.

Mabema har en produkt som mäter vedvolym av virkestravar på lastbilar. Systemet är byggt på att en bildbehandlingsalgoritm hittar silhuetterna av stockarna på renderade bilder av lastbilstravar. Arbetsgivaren är inte helt nöjd med prestandan av algoritmen och vill utreda om djupinlärning kan förbättra resultatet. Detta arbete undersöker hur diameterfördelningen i varje trave kan estimeras med hjälp av djupinlärning och objektdetektering i synnerhet. Två metoder granskas, den ena hanterar problemet abstrakt med djup regression medan den andra metoden går in i detalj och nyttjar objektigenkänning för att hitta stockändar. Arbetet utvärderar även möjliheterna att träna dessa modeller baserat på data från fysiska simulationer. Det visar sig vara användbart att nyttja syntetisk data för träning och med transfer learning lyckas de syntetiska modellen uppnå kraven Biometria ställer på automatiserad diameterberäkning. Med objektdetektering visar det sig också gå att uppnå samma prestanda som arbetsgivarens algoritm med en bättre stocksökning tre gånger så snabbt eller snabbare.