The author investigates the performance of a time series based approach in predicting the risk of patients abandoning treatment in a treatment program for childhood obesity. The time series based approach is compared and contrasted to an approach based on static features (which has been applied in similar problems). Four machine learning models are constructed; one ‘Main model’ using both time series forecasting and three ‘reference models’ created by removing or exchanging parts of the main model to test the performance of using only time series forecasting or only static features in the prediction. The main model achieves an ROC-AUC of 0.77 on the data set. ANOVA testing is used to determine whether the four models perform differently. A difference cannot be verified at the significance level of 0.05, and thus, the author concludes that the project cannot show either an advantage or a disadvantage to employing a time series based approach over static features in this problem. / Författaren jämför modeller baserade på tidsserieförutsägelser med modeller baserade på statiska, fasta värden, till syfte att identifera patienter som riskerar att lämna ett behandlingsprogram för barnfetma. Fyra maskininlärningsmodeller konstrueras, en ‘Huvudmodell’ som använder sig av både tidsserieförutsägelser och statiska värden, och tre modeller som bryter ut delar av huvudmodellen för undersöka beteendet i modeller baserade enbart på statiska värden respektive enbart baserade på tidsserieförutsägelser. Huvudmodellen uppnår ROC-AUC0.77 på datasetet. ANOVA(variansanalys) används för att avgöra huruvida de fyra modellernas resultat skiljer sig, och en skillnad kan ej verieras vid P = 0:05. Följaktligen drar författaren slutsatsen att projektet inte har kunnat visa vare sig en signifikant fördel eller nackdel med att använda sig av tidsserieförutsägelser inom den aktuella problemdomänen.

The Transformer, an attention-based deep learning architecture, has shown promising capabilities in both Natural Language Processing and Computer Vision. Recently, it has also been applied to time series classification, which has traditionally used statistical methods or the Gated Recurrent Unit (GRU). The aim of this project was to apply multivariate time series classification to evaluate Transformer-based models, in comparison with the traditional GRUs. The evaluation was done within the problem of startup success prediction at a venture and private equity firm called EQT. Four different Machine Learning (ML) models – the Univariate GRU, Multivariate GRU, Transformer Encoder, and an already existing implementation, the Time Series Transformer (TST) – were benchmarked using two public datasets and the EQT dataset which utilized an investor-centric data split. The results suggest that the TST is the best-performing model on EQT’s dataset within the scope of this project, with a 47% increase in performance – measured by the Area Under the Curve (AUC) metric – compared to the Univariate GRU, and a 12% increase compared to the Multivariate GRU. It was also the best, and third-best, performing model on the two public datasets. Additionally, the model also demonstrated the highest training stability out of all four models, and 15 times shorter training times than the Univariate GRU. The TST also presented several potential qualitative advantages such as utilizing its embeddings for downstream tasks, an unsupervised learning technique, higher explainability, and improved multi-modal compatibility. The project results, therefore, suggest that the TST is a viable alternative to the GRU architecture for multivariate time series classification within the investment domain. With its performance, stability, and added benefits, the TST is certainly worth considering for time series modeling tasks. / Transformern är en attention-baserad arkitektur skapad för djupinlärning som har demonsterat lovande kapacitet inom både naturlig språkbehandling och datorseende. Nyligen har det även tillämpats på tidsserieklassificering, som traditionellt har använt statistiska metoder eller GRU. Syftet med detta projekt var att tillämpa multivariat tidsserieklassificering för att utvärdera transformer-baserade modeller, i jämförelse med de traditionella GRUerna. Jämförelsen gjordes inom problemet med att klassificera vilka startup-företag som är potentiellt framgångsrika eller inte, och gjordes på ett risk- och privatkapitalbolag som heter EQT. Fyra olika maskininlärningsmodeller – Univariat GRU, Multivariat GRU, Transformer Encoder och en redan existerande implementering, TST – jämfördes med hjälp av två offentliga datamängder och EQT-datamängden som använde sig av en investerarcentrerad datauppdelning. Resultaten tyder på att TST är den modellen som presterar bäst på EQT:s datauppsättning inom ramen för detta projekt, med en 47% ökning i prestanda – mätt med AUC – jämfört med den univariata GRUn och en ökning på 12% jämfört med den multivariata GRUn. Det var också den bäst och tredje bäst presterande modellen på de två offentliga datamängderna. Modellen visade även den högsta träningsstabiliteten av alla fyra modellerna och 15 gånger kortare träningstider än den univariata GRUn. TST visade även flera potentiella kvalitativa fördelar som att använda dess inbäddningar för nedströmsuppgifter, en oövervakad inlärningsteknik, högre förklarabarhet och förbättrad multimodal kompatibilitet. Projektresultaten tyder därför på att TST är ett gångbart alternativ till GRUarkitekturen för multivariat tidsserieklassificering inom investeringsdomänen. Med sin prestanda, stabilitet och extra fördelar är TST verkligen värt att överväga för tidsseriemodelleringsproblem.

Semantic labeling is becoming more and more popular among researchers in computer vision and machine learning. Many applications, such as autonomous driving, tracking, indoor navigation, augmented reality systems, semantic searching, medical imaging are on the rise, requiring more accurate and efficient segmentation mechanisms. In recent years, deep learning approaches based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have dramatically emerged as the dominant paradigm for solving many problems in computer vision and machine learning. The main focus of this thesis is to investigate robust approaches that can tackle the challenging semantic labeling tasks including semantic instance segmentation and scene understanding. In the first approach, we convert the classic variational Level Set method to a learnable deep framework by proposing a novel definition of contour evolution named Recurrent Level Set (RLS). The proposed RLS employs Gated Recurrent Units to solve the energy minimization of a variational Level Set functional. The curve deformation processes in RLS is formulated as a hidden state evolution procedure and is updated by minimizing an energy functional composed of fitting forces and contour length. We show that by sharing the convolutional features in a fully end-to-end trainable framework, RLS is able to be extended to Contextual Recurrent Level Set (CRLS) Networks to address semantic segmentation in the wild problem. The experimental results have shown that our proposed RLS improves both computational time and segmentation accuracy against the classic variational Level Set-based methods whereas the fully end-to-end system CRLS achieves competitive performance compared to the state-of-the-art semantic segmentation approaches on PAS CAL VOC 2012 and MS COCO 2014 databases. The second proposed approach, Contextual Recurrent Residual Networks (CRRN), inherits all the merits of sequence learning information and residual learning in order to simultaneously model long-range contextual infor- mation and learn powerful visual representation within a single deep network. Our proposed CRRN deep network consists of three parts corresponding to sequential input data, sequential output data and hidden state as in a recurrent network. Each unit in hidden state is designed as a combination of two components: a context-based component via sequence learning and a visualbased component via residual learning. That means, each hidden unit in our proposed CRRN simultaneously (1) learns long-range contextual dependencies via a context-based component. The relationship between the current unit and the previous units is performed as sequential information under an undirected cyclic graph (UCG) and (2) provides powerful encoded visual representation via residual component which contains blocks of convolution and/or batch normalization layers equipped with an identity skip connection. Furthermore, unlike previous scene labeling approaches [1, 2, 3], our method is not only able to exploit the long-range context and visual representation but also formed under a fully-end-to-end trainable system that effectively leads to the optimal model. In contrast to other existing deep learning networks which are based on pretrained models, our fully-end-to-end CRRN is completely trained from scratch. The experiments are conducted on four challenging scene labeling datasets, i.e. SiftFlow, CamVid, Stanford background, and SUN datasets, and compared against various state-of-the-art scene labeling methods.

Today, we are living in a data-driven world. Due to a surge in data generation, there is a need for efficient and accurate techniques to analyze data. One such kind of data which is needed to be analyzed are text reviews given for movies. Rather than classifying the reviews as positive or negative, we will classify the sentiment of the reviews on the scale of one to ten. In doing so, we will compare two recurrent neural network algorithms Long short term memory(LSTM) and Gated recurrent unit(GRU). The main objective of this study is to compare the accuracies of LSTM and GRU models. For training models, we collected data from two different sources. For filtering data, we used porter stemming and stop words. We coupled LSTM and GRU with the convolutional neural networks to increase the performance. After conducting experiments, we have observed that LSTM performed better in predicting border values. Whereas, GRU predicted every class equally. Overall GRU was able to predict multiclass text data of movie reviews slightly better than LSTM. GRU was computationally expansive when compared to LSTM.

No description available.

Cette thèse porte sur les modèles génératifs en apprentissage automatique. Deux nouveaux modèles basés sur les réseaux de neurones y sont proposés. Le premier modèle possède une représentation interne où une certaine structure a été imposée afin d’ordonner les caractéristiques apprises. Le deuxième modèle parvient à exploiter la structure topologique des données observées, et d’en tenir compte lors de la phase générative. Cette thèse présente également une des premières applications de l’apprentissage automatique au problème de la tractographie du cerveau. Pour ce faire, un réseau de neurones récurrent est appliqué à des données de diffusion afin d’obtenir une représentation des fibres de la matière blanche sous forme de séquences de points en trois dimensions.

Multi-Task Learning is today an interesting and promising field which many mention as a must for achieving the next level advancement within machine learning. However, in reality, Multi-Task Learning is much more rarely used in real-world implementations than its more popular cousin Transfer Learning. The questionis why that is and if Multi-Task Learning outperforms its Single-Task counterparts. In this thesis different Multi-Task Learning architectures were utilized in order to build a model that can handle labeling real technical issues within two categories. The model faces a challenging imbalanced data set with many labels to choose from and short texts to base its predictions on. Can task-sharing be the answer to these problems? This thesis investigated three Multi-Task Learning architectures and compared their performance to a Single-Task model. An authentic data set and two labeling tasks was used in training the models with the method of supervised learning. The four model architectures; Single-Task, Multi-Task, Cross-Stitched and the Shared-Private, first went through a hyper parameter tuning process using one of the two layer options LSTM and GRU. They were then boosted by auxiliary tasks and finally evaluated against each other.

As an efficient energy storage technology, lithium-ion batteries play a key role in the ongoing electrification of the mobility sector. However, the required modelbased design process, including hardware in the loop solutions, demands precise battery models. In this work, an encoder-decoder model framework based on recurrent neural networks is developed and trained directly on unstructured battery data to replace time consuming characterisation tests and thus simplify the modelling process. A manifold pseudo-random bit stream dataset is used for model training and validation. A mean percentage error (MAPE) of 0.30% for the test dataset attests the proposed encoder-decoder model excellent generalisation capabilities. Instead of the recursive one-step prediction prevalent in the literature, the stage-wise trained encoder-decoder framework can instantaneously predict the battery voltage response for 2000 time steps and proves to be 120 times more time-efficient on the test dataset. Accuracy, generalisation capability and time efficiency of the developed battery model enable a potential online anomaly detection, power or range prediction. The fact that, apart from the initial voltage level, the battery model only relies on the current load as input and thus requires no estimated variables such as the state-of-charge (SOC) to predict the voltage response holds the potential of a battery ageing independent LIB modelling based on raw BMS signals. The intrinsically ageingindependent battery model is thus suitable to be used as a digital battery twin in virtual experiments to estimate the unknown battery SOH on purely BMS data basis.

The thesis two-folded problem aim was to identify and evaluate candidate Machine Learning (ML) methods and performance methods, for predicting the weekly number of covid-19 infections. The two-folded problem aim was created from studying public health studies where several challenges were identified. One challenge identified was the lack of using sophisticated and hybrid ML methods in the public health research area. In this thesis a comparison of ML methods for predicting the number of covid-19 weekly infections has been performed. A dataset taken from the Public Health Agency in Sweden consisting of 101weeks divided into a 60 % training set and a 40% testing set was used in the evaluation. Five candidate ML methods have been investigated in this thesis called Support Vector Regressor (SVR), Long Short Term Memory (LSTM), Gated Recurrent Network (GRU), Bidirectional-LSTM (BI-LSTM) and LSTM-Convolutional Neural Network (LSTM-CNN). These methods have been evaluated based on three performance measurements called Root Mean Squared Error (RMSE), Mean Absolute Error (MAE) and R2. The evaluation of these candidate ML resulted in the LSTM-CNN model performing the best on RMSE, MAE and R2.

Achieving appropriate and natural backchanneling behavior in social robots remains a challenge in Human-Robot Interaction (HRI). This thesis addresses this issue by utilizing methods from Safe Reinforcement Learning in particular shielding to improve social robot backchanneling behavior. The aim of the study is to develop and implement a safety shield that guarantees appropriate backchanneling. In order to achieve that, a Recurrent Neural Network (RNN) is trained on a human-human conversational dataset. Two agents are built; one uses a random algorithm to backchannel and another uses shields on top of its algorithm. The two agents are tested using a recorded human audio, and later evaluated in a between-subject user study with 41 participants. The results did not show any statistical significance between the two conditions, for the chosen significance level of α < 0.05. However, we observe that the agent with shield had a better listening behavior, more appropriate backchanneling behavior and missed less backchanneling opportunities than the agent without shields. This could indicate that shields have a positive impact on the robot’s behavior. We discuss potential explanations for why we did not obtain statistical significance and shed light on the potential for further exploration. / Att uppnå lämpligt och naturligt upbbackningsbeteende i sociala robotar är fortfarande en utmaning i Människa-Robot Interaktion (MRI). Den här avhandlingen tar upp detta problem genom att använda metoder från säker förstärkningsinlärning i synnerhet avskärmning för att förbättra sociala robotars upbbackningsbeteende. Syftet med studien är att utveckla och implementera en säkerhetsavskärmning som garanterar lämplig upbbackning. För att uppnå det, tränas ett återkommande neuralt nätverk på en människa-människa konversationsdatamängd. Två agenter byggs; en använder en slumpmässig algoritm för att upbbacka och en annan använder avskärmninng ovanpå sin algoritm. De två agenterna testas med hjälp av ett inspelat mänskligt ljud och utvärderas senare i en användarstudie med 41 deltagare. Resultaten visade inte någon statistisk signifikans mellan de två skicken, för den valda signifikansnivån < 0, 05. Vi observerar dock att agenten med avskärmning hade ett bättre lyssningsbeteende, mer lämplig upbbackningsbeteende och missade mindre upbbacknings-möjligheter än agenten utan avskärmning. Detta kan indikera att avskärmning har en positiv inverkan på robotarnas beteende. Vi diskuterar potentiella förklaringar till varför vi inte fick statistisk signifikans och belyser potentialen för ytterligare utforskning.