Autonomous vehicles rely on sensors for a clear understanding of the environment and in a heavy duty truck, the sensors are placed at multiple locations like the cabin, chassis and the trailer in order to increase the field of view and reduce the blind spot area. Usually, these sensors perform best when they are stationary relative to the ground, hence large and fast movements, which are quite common in a truck, may lead to performance reduction, erroneous data or in the worst case, a sensor failure. This enforces a need to validate the sensors before using them for making life-critical decisions. This thesis proposes Domain Adaptation as one of the strategies to co-validate Inertial Measurement Unit (IMU) sensors. The proposed Generative Adversarial Network (GAN) based framework predicts the data of one IMU using other IMUs in the truck by implicitly learning the internal dynamics. This prediction model along with other sensor fusion strategies would be used by the supervising system to validate the IMUs in real-time. Through data collected from real-world experiments, it is shown that the proposed framework is able to accurately transform raw IMU sequences across domains. A further comparison is made between Long Short Term Memory (LSTM) and WaveNet based architectures to show the superiority of WaveNets in terms of performance and computational efficiency. / Autonoma fordon förlitar sig på sensorer för att skapa en bild av omgivningen. På en tung lastbil placeras sensorerna på multipla ställen, till exempel på hytten, chassiet och på trailern för att öka siktfältet och för att minska blinda områden. Vanligtvis presterar sensorerna som bäst när de är stationära i förhållande till marken, därför kan stora och snabba rörelser, som är vanliga på en lastbil, leda till nedsatt prestanda, felaktig data och i värsta fall fallerande sensorer. På grund av detta så finns det ett stort behov av att validera sensordata innan det används för kritiskt beslutsfattande. Den här avhandlingen föreslår domänadaption som en av de strategier för att samvalidera Tröghetsmätningssensorer (IMU-sensorer). Det föreslagna Generative Adversarial Network (GAN) baserade ramverket förutspår en Tröghetssensors data genom att implicit lära sig den interna dynamiken från andra Tröghetssensorer som är monterade på lastbilen. Den här prediktionsmodellen kombinerat med andra sensorfusionsstrategier kan användas av kontrollsystemet för att i realtid validera Tröghetssensorerna. Med hjälp av data insamlat från verkliga experiment visas det att det föreslagna ramverket klarar av att med hög noggrannhet konvertera obehandlade Tröghetssensor-sekvenser mellan domäner. Ytterligare en undersökning mellan Long Short Term Memory (LSTM) och WaveNet-baserade arkitekturer görs för att visa överlägsenheten i WaveNets när det gäller prestanda och beräkningseffektivitet.

Within demand forecasting, and specifically within the field of e-commerce, the provided data often contains erratic behaviours which are difficult to explain. This induces contradictions to the common assumptions within classical approaches for time series analysis. Yet, classical and naive approaches are still commonly used. Machine learning could be used to alleviate such problems. This thesis evaluates four models together with Swedish fin-tech company QLIRO AB. More specifically, a MLR (Multiple Linear Regression) model, a classic Box-Jenkins model (SARIMAX), an XGBoost model, and a LSTM-network (Long Short-Term Memory). The provided data consists of aggregated total daily reservations by e-merchants within the Nordic market from 2014. Some data pre processing was required and a smoothed version of the data set was created for comparison. Each model was constructed according to their specific requirements but with similar feature engineering. Evaluation was then made on a monthly level with a forecast horizon of 30 days during 2021. The results shows that both the MLR and the XGBoost provides the most consistent results together with perks for being easy to use. After these two, the LSTM-network showed the best results for November and December on the original data set but worst overall. Yet it had good performance on the smoothed data set and was then comparable to the first two. The SARIMAX was the worst performing of all the models considered in this thesis and was not as easy to implement. / Inom efterfrågeprognoser, och specifikt inom området e-handel, innehåller den tillhandahållna informationen ofta oberäkneliga beteenden som är svåra att förklara. Detta motsäger vanliga antaganden inom tidsserier som används för de mer klassiska tillvägagångssätten. Ändå är klassiska och naiva metoder fortfarande vanliga. Maskininlärning skulle kunna användas för att lindra sådana problem. Detta examensarbete utvärderar fyra modeller tillsammans med det svenska fintechföretaget QLIRO AB. Mer specifikt en MLR-modell (Multiple Linear Regression), en klassisk Box-Jenkins-modell (SARIMAX), en XGBoost-modell och ett LSTM-nätverk (Long Short-Term Memory). Den tillhandahållna informationen består av aggregerade dagliga reservationer från e-handlare inom den nordiska marknaden från 2014. Viss dataförbehandling krävdes och en utjämnad version av datamängden skapades för jämförelse. Varje modell konstruerades enligt deras specifika krav men med liknande \textit{feature engineering}. Utvärderingen gjordes sedan på månadsnivå med en prognoshorisont på 30 dagar under 2021. Resultaten visar att både MLR och XGBoost ger de mest pålitliga resultaten tillsammans med fördelar som att vara lätta att använda. Efter dessa visar LSTM-nätverket de bästa resultaten för november och december på den ursprungliga datamängden men sämst totalt sett. Ändå visar den god prestanda på den utjämnade datamängden och var sedan jämförbar med de två första modellerna. SARIMAX var den sämst presterande av alla jämförda modeller och inte lika lätt att implementera.

Estimating each marketing channel’s impact on conversion can help advertisers develop strategies and spend their marketing budgets optimally. This problem is often referred to as attribution modelling, and it is gaining increasing attention in both the industry and academia as access to online tracking data improves. Focusing on achieving higher predictive performance, the Long Short- Term Memory (LSTM) architecture is currently trending as a data-driven solution to attribution modelling. However, such deep neural networks have been criticised for being difficult to interpret. Interpretability is critical, since channel attributions are generally obtained by studying how a model makes a binary conversion prediction given a sequence of clicks or views of ads in different channels. Therefore, this degree project studies and compares the quality of LSTM attributions, calculated with SHapleyAdditive exPlanations (SHAP), attention and fractional scores to three baseline models. The fractional score is the mean difference in a model’s predicted conversion probability with and without a channel. Furthermore, a synthetic data generator based on a Poisson process is developed and validated against real data to measure attribution quality as the Mean Absolute Error (MAE) between calculated attributions and the true causal relationships between channel clicks and conversions. The experimental results demonstrate that the quality of attributions is not unambiguously reflected by the predictive performance of LSTMs. In general, it is not possible to assume a high attribution quality solely based on high predictive performance. For example, all models achieve ~82% accuracy on real data, whereas LSTM Fractional and SHAP produce the lowest attribution quality of 0:0566 and 0:0311 MAE respectively. This can be compared to an improved MAE of 0:0058, which is obtained with a Last-Touch Attribution (LTA) model. The attribution quality also varies significantly depending on which attribution calculation method is used for the LSTM. This suggests that the ongoing quest for improved accuracy may be questioned and that it is not always justified to use an LSTM when aiming for high quality attributions. / Genom att estimera påverkan varje marknadsföringskanal har på konverteringar, kan annonsörer utveckla strategier och spendera sina marknadsföringsbudgetar optimalt. Det här kallas ofta attributionsmodellering och det får alltmer uppmärksamhet i både näringslivet och akademin när tillgången till spårningsinformation ökar online. Med fokus på att uppnå högre prediktiv prestanda är Long Short-Term Memory (LSTM) för närvarande en populär datadriven lösning inom attributionsmodellering. Sådana djupa neurala nätverk har dock kritiserats för att vara svårtolkade. Tolkningsbarhet är viktigt, då kanalattributioner generellt fås genom att studera hur en modell gör en binär konverteringsprediktering givet en sekvens av klick eller visningar av annonser i olika kanaler. Det här examensarbetet studerar och jämför därför kvaliteten av en LSTMs attributioner, beräknade med SHapley Additive exPlanations (SHAP), attention och fractional scores mot tre grundmodeller. Fractional scores beräknas som medelvärdesdifferensen av en modells predikterade konverteringssannolikhet med och utan en viss kanal. Därutöver utvecklas en syntetisk datagenerator baserad på en Poissonprocess, vilken valideras mot verklig data. Generatorn används för att kunna mäta attributionskvalitet som Mean Absolute Error (MAE) mellan beräknade attributioner och de verkliga kausala sambanden mellan kanalklick och konverteringar. De experimentella resultaten visar att attributionskvaliteten inte entydigt avspeglas av en LSTMs prediktiva prestanda. Det är generellt inte möjligt att anta en hög attributionskvalitet enbart baserat på en hög prediktiv prestanda. Alla modeller uppnår exempelvis ~82% prediktiv träffsäkerhet på verklig data, medan LSTM Fractional och SHAP ger den lägsta attributionskvaliteten på 0:0566 respektive 0:0311 MAE. Det här kan jämföras mot en förbättrad MAE på 0:0058, som erhålls med en Last-touch-modell. Kvaliteten på attributioner varierar också signifikant beroende på vilket metod för attributionsberäkning som används för LSTM. Det här antyder att den pågående strävan efter högre prediktiv träffsäkerhet kan ifrågasättas och att det inte alltid är berättigat att använda en LSTM när attributioner av hög kvalitet eftersträvas.

This thesis focuses on statistical and machine learning methods designed for sequential and repeated measurement data. We start off by considering the classic general linear model (MANOVA) followed by its generalization, the growth curve model (GMANOVA), designed for analysis of repeated measurement data. By considering a binary classification problem of normal data together with the corresponding maximum likelihood estimators for the growth curve model, we demonstrate how a classification rule based on linear discriminant analysis can be derived which can be used for repeated measurement data in a meaningful way. We proceed to the topics of neural networks which serve as our second method of classification. The reader is introduced to classic neural networks and relevant subtopics are discussed. We present a generalization of the classic neural network model to the recurrent neural network model and the LSTM model which are designed for sequential data. Lastly, we present three types of data sets with an total of eight cases where the discussed classification methods are tested. / Den här uppsatsen introducerar klassificeringsmetoder skapade för data av typen upprepade mätningar och sekventiell data. Den klassiska MANOVA modellen introduceras först som en grund för den mer allmäna tillväxtkurvemodellen(GMANOVA), som i sin tur används för att modellera upprepade mätningar på ett meningsfullt sätt. Under antagandet av normalfördelad data så härleds en binär klassificeringsmetod baserad på linjär diskriminantanalys, som tillsammans med maximum likelihood-skattningar från tillväxtkurvemodellen ger en binär klassificeringsregel för data av typen upprepade mätningarn. Vi fortsätter med att introducera läsaren för klassiska neurala nätverk och relevanta ämnen diskuteras. Vi generaliserar teorin kring neurala nätverk till typen "recurrent" neurala nätverk och LSTM som är designade för sekventiell data. Avslutningsvis så testas klassificeringsmetoderna på tre typer av data i totalt åtta olika fall.

Two mm-wave frequency modulated continuous wave (FMCW) radars were combined with a recurrent neural network (RNN) to perform fall detection. The purpose was to find methods to implement a multi-radar setup for healthcare monitoring and to study the resulting models’ resilience to interference and other obstacles, such as re-arranging the radars in the room. Single-board computers (SBCs) controlled the radars to record and transfer data over Ethernet to a PC. The Ethernet connection also allowed synchronization with the network time protocol (NTP), which was necessary to put the data from the two sensors in correspondence. The proposed RNN used two bidirectional long-short term memory (Bi-LSTM) layers with L2-regularization and dropout layers. It had an overall accuracy of 95.15% and 98.11% recall with a test set. Performance in live testing varied with different arrangements, with an accuracy of 98% with the radars along the same wall, 94% with the radars diagonally, and 90% with an alternative arrangement that the RNN model had not seen during training. However, the latter arrangement resulted in a recall of 95.7%, with false alarms reducing the overall performance. In conclusion, the model performed adequately for fall detection, even with different radar arrangements but could still be sensitive to interference. / Två millimetervågs-radarsystem av typen frequency modulated continuous wave (FMCW) kombinerades för att med hjälp av ett recurrent neural network (RNN) utföra falldetektering. Syftet var att finna metoder för att implementera en multiradarplatform för hälsoövervakning samt att studera de resulterande modellernas tolerans mot interferens och andra hinder så som att radarsystemen placeras på olika sätt i rummet. Enkortsdatorer kontrollerade radarsystemen för att kunna spela in och överföra data över Ethernet till en PC. Ethernetanslutningen möjliggjorde även synkronisering över network time protocol (NTP), vilket var nödvändigt för att sammanlänka datan från de båda sensorerna. Det föreslagna RNN:et använde två dubbelriktade (bidirectional) long-short term memory (Bi-LSTM) lager med L2-regularisering och dropout-lager. Det hade en total noggrannhet på 95.15% och 98.11% recall med ett test-set. Prestandan vid testning i drift varierade beroende på olika uppställningar av radarmodulerna, med en noggrannhet på 98% då de placerades längs samma vägg, 94% då de placerades diagonalt och 90% vid en alternativ uppställning som RNN-modellen inte hade sett när den tränades. Det senare resulterade dock i 95.7% recall, där falsklarm var den främsta felkällan. Sammanfattningsvis presterade modellen bra för falldetektering, även med olika uppställningar, men den verkar fortfarande vara känslig för interferens.

This study aims to investigate the implementation of an AI model that predicts customer purchases, in the telecom industry. The thesis also outlines how such an AI model can assist decision-making in marketing strategies. It is concluded that designing the AI model by following a Recurrent Neural Network (RNN) architecture with a Long Short-Term Memory (LSTM) layer, allow for a successful implementation with satisfactory model performances. Stepwise instructions to construct such model is presented in the methodology section of the study. The RNN-LSTM model further serves as an assisting tool for marketers to assess how a consumer’s website behavior affect their purchase behavior over time, in a quantitative way - by observing what the authors refer to as the Customer Purchase Propensity Journey (CPPJ). The firm empirical basis of CPPJ, can help organizations improve their allocation of marketing resources, as well as benefit the organization’s online presence by allowing for personalization of the customer experience. / Denna studie undersöker implementeringen av en AI-modell som förutspår kunders köp, inom telekombranschen. Studien syftar även till att påvisa hur en sådan AI-modell kan understödja beslutsfattande i marknadsföringsstrategier. Genom att designa AI-modellen med en Recurrent Neural Network (RNN) arkitektur med ett Long Short-Term Memory (LSTM) lager, drar studien slutsatsen att en sådan design möjliggör en framgångsrik implementering med tillfredsställande modellprestation. Instruktioner erhålls stegvis för att konstruera modellen i studiens metodikavsnitt. RNN-LSTM-modellen kan med fördel användas som ett hjälpande verktyg till marknadsförare för att bedöma hur en kunds beteendemönster på en hemsida påverkar deras köpbeteende över tiden, på ett kvantitativt sätt - genom att observera det ramverk som författarna kallar för Kundköpbenägenhetsresan, på engelska Customer Purchase Propensity Journey (CPPJ). Den empiriska grunden av CPPJ kan hjälpa organisationer att förbättra allokeringen av marknadsföringsresurser, samt gynna deras digitala närvaro genom att möjliggöra mer relevant personalisering i kundupplevelsen.

Long short-term volume forecasting is essential for companies regarding their logistics service operations. It is crucial for logistic companies to predict the volumes of goods that will be delivered to various centers at any given day, as this will assist in managing the efficiency of their business operations. This research aims to create a forecasting model for outbound logistics volumes by utilizing design science research methodology in building 3 machine-learning models and evaluating the performance of the models . The dataset is provided by Tetra Pak AB, the World's leading food processing and packaging solutions company,. Research methods were mainly quantitative, based on statistical data and numerical calculations. Three algorithms were implemented: which are encoder–decoder networks based on Long Short-Term Memory (LSTM), Convolutional Long Short-Term Memory (ConvLSTM), and Convolutional Neural Network Long ShortTerm Memory (CNN-LSTM). Comparisons are made with the average Root Mean Square Error (RMSE) for six distribution centers (DC) of Tetra Pak. Results obtained from encoder–decoder networks based on LSTM are compared to results obtained by encoder–decoder networks based on ConvLSTM and CNN-LSTM. The three algorithms performed very well, considering the loss of the Train and Test with our multivariate time series dataset. However, based on the average score of the RMSE, there are slight differences between algorithms for all DCs.

Under recent years the gambling industry has been moving towards providing their customer the possibility to gamble online instead of visiting a physical location. Aggressive marketing, fast growth and a multitude of actors within the market have resulted in a spike of customers who have developed a gambling problem. Decision makers are trying to fight back by regulating markets in order to make the companies take responsibility and work towards preventing these problems. One method of working proactively in this regards is to identify vulnerable customers before they develop a destructive habit. In this work a novel method of predicting customers that have a higher risk in regards to gambling-related problems is explored. More concretely, a recurrent neural network with long short-term memory cells is created to process raw behaviour data that are aggregated on a daily basis to classify them as high-risk or not. Supervised training is used in order to learn from historical data, where the usage of permanent self-exclusions due to gambling related problems defines problem gamblers. The work consists of: obtain a local optimal configuration of the network which enhances the performance for identifying problem gam- blers who favour the casino section over sports section, and analyze the model to provide insights in the field. This project was carried out together with LeoVegas Mobile Gaming Group. The group offers both online casino games and sports booking in a number of countries in Europe. This collaboration made both data and expertise within the industry accessible to perform this work. The company currently have a model in production to perform these predictions, but want to explore other approaches. The model that has been developed showed a significant increase in performance compared to the one that is currently used at the company. Specifically, the precision and recall which are two metrics important for a two class classification model, increased by 37% and 21% respectively. Using raw time series data, instead of aggregated data increased the responsiveness regarding customers change in behaviour over time. The model also scaled better with more history compared to the current model, which could be a result of the nature of a recurrent network compared to the current model used.

Sequences are ubiquitous in the domain of biology. One of the current best machine learning techniques for analysing sequences is the long short-term memory (LSTM) network. Owing to significant barriers to adoption in biology, focussed efforts are required to realize the use of LSTMs in practice. Thus, the aim of this work is to improve the state of LSTMs for biology, and we focus on biological tasks pertaining to physiological signals, peripheral neural signals, and molecules. This goal drives the three subplots in this thesis: biological applications, visualizations, and extensions. We start by demonstrating the utility of LSTMs for biological applications. On two new physiological-signal datasets, LSTMs were found to outperform hidden Markov models. LSTM-based models, implemented by other researchers, also constituted the majority of the best performing approaches on publicly available medical datasets. However, even if these models achieve the best performance on such datasets, their adoption will be limited if they fail to indicate when they are likely mistaken. Thus, we demonstrate on medical data that it is straightforward to use LSTMs in a Bayesian framework via dropout, providing model predictions with corresponding uncertainty estimates. Another dataset used to show the utility of LSTMs is a novel collection of peripheral neural signals. Manual labelling of this dataset is prohibitively expensive, and as a remedy, we propose a sequence-to-sequence model regularized by Wasserstein adversarial networks. The results indicate that the proposed model is able to infer which actions a subject performed based on its peripheral neural signals with reasonable accuracy. As these LSTMs achieve state-of-the-art performance on many biological datasets, one of the main concerns for their practical adoption is their interpretability. We explore various visualization techniques for LSTMs applied to continuous-valued medical time series and find that learning a mask to optimally delete information in the input provides useful interpretations. Furthermore, we find that the input features looked for by the LSTM align well with medical theory. For many applications, extensions of the LSTM can provide enhanced suitability. One such application is drug discovery -- another important aspect of biology. Deep learning can aid drug discovery by means of generative models, but they often produce invalid molecules due to their complex discrete structures. As a solution, we propose a version of active learning that leverages the sequential nature of the LSTM along with its Bayesian capabilities. This approach enables efficient learning of the grammar that governs the generation of discrete-valued sequences such as molecules. Efficiency is achieved by reducing the search space from one over sequences to one over the set of possible elements at each time step -- a much smaller space. Having demonstrated the suitability of LSTMs for biological applications, we seek a hardware efficient implementation. Given the success of the gated recurrent unit (GRU), which has two gates, a natural question is whether any of the LSTM gates are redundant. Research has shown that the forget gate is one of the most important gates in the LSTM. Hence, we propose a forget-gate-only version of the LSTM -- the JANET -- which outperforms both the LSTM and some of the best contemporary models on benchmark datasets, while also reducing computational cost.

This study is an investigation if a recurrent long short-term memory (LSTM) based neural network can be used to estimate the battery capacity in electrical cars. There is an enormous interest in finding the underlying reasons why and how Lithium-ion batteries ages and this study is a part of this broader question. The research questions that have been answered are how well a LSTM model estimates the battery capacity, how the LSTM model is performing compared to a linear model and what parameters that are important when estimating the capacity. There have been other studies covering similar topics but only a few that has been performed on a real data set from real cars driving. With a data science approach, it was discovered that the LSTM model indeed is a powerful model to use for estimation the capacity. It had better accuracy than a linear regression model, but the linear regression model still gave good results. The parameters that implied to be important when estimating the capacity were logically related to the properties of a Lithium-ion battery.En studie över hur väl ett återkommande neuralt nätverk kan estimera kapaciteten hos Litium-ion batteri hos elektroniska fordon, när en en datavetenskaplig strategi har använts.