This thesis concerns the application of different artificial neural network architectures on the classification of multivariate accelerometer time series data into activity classes such as sitting, lying down, running, or walking. There is a strong correlation between increased health risks in children and their amount of daily screen time (as reported in questionnaires). The dependency is not clearly understood, as there are no such dependencies reported when the sedentary (idle) time is measured objectively. Consequently, there is an interest from the medical side to be able to perform such objective measurements. To enable large studies the measurement equipment should ideally be low-cost and non-intrusive. The report investigates how well these movement patterns can be distinguished given a certain measurement setup and a certain network structure, and how well the networks generalise to noisier data. Recurrent neural networks are given extra attention among the different networks, since they are considered well suited for data of sequential nature. Close to state-of-the-art results (95% weighted F1-score) are obtained for the tasks with 4 and 5 classes, which is notable since a considerably smaller number of sensors is used than in the previously published results. Another contribution of this thesis is that a new labeled dataset with 12 activity categories is provided, consisting of around 6 hours of recordings, comparable in number of samples to benchmarking datasets. The data collection was made in collaboration with the Department of Public Health at Karolinska Institutet. / Inom ramen för uppsatsen testas hur väl rörelsemönster kan urskiljas ur accelerometerdatamed hjälp av den gren av maskininlärning som kallas djupinlärning; där djupa artificiellaneurala nätverk av noder funktionsapproximerar mappandes från domänen av sensordatatill olika fördefinerade kategorier av aktiviteter så som gång, stående, sittande eller liggande.Det finns ett intresse från den medicinska sidan att kunna mäta fysisk aktivitet objektivt,bland annat eftersom det visats att det finns en korrelation mellan ökade hälsorisker hosbarn och deras mängd daglig skärmtid. Denna typ av mätningar ska helst kunna göras medicke-invasiv utrustning till låg kostnad för att kunna göra större studier.Enklare nätverksarkitekturer samt återimplementeringar av bästa möjliga teknik inomområdet Mänsklig aktivitetsigenkänning (HAR) testas både på ett benchmarkingdataset ochpå egeninhämtad data i samarbete med Institutet för Folkhälsovetenskap på Karolinska Institutetoch resultat redovisas för olika val av möjliga klassificeringar och olika antal dimensionerper mätpunkt. De uppnådda resultaten (95% F1-score) på ett 4- och 5-klass-problem ärjämförbara med de bästa tidigare publicerade resultaten för aktivitetsigenkänning, vilket äranmärkningsvärt då då betydligt färre accelerometrar har använts här än i de åsyftade studierna.Förutom klassificeringsresultaten som redovisas bidrar det här arbetet med ett nyttinhämtat och kategorimärkt dataset; KTH-KI-AA. Det är jämförbart i antal datapunkter medspridda benchmarkingdataset inom HAR-området.

<p dir="ltr">The research focuses on improving natural language processing (NLP) models by integrating the hierarchical structure of language, which is essential for understanding and generating human language. The main contributions of the study are:</p><ol><li><b>Hierarchical RNN Model:</b> Development of a deep Recurrent Neural Network model that captures both explicit and implicit hierarchical structures in language.</li><li><b>Hierarchical Attention Mechanism:</b> Use of a multi-level attention mechanism to help the model prioritize relevant information at different levels of the hierarchy.</li><li><b>Latent Indicators and Efficient Training:</b> Integration of latent indicators using the Expectation-Maximization algorithm and reduction of computational complexity with Bootstrap sampling and layered training strategies.</li><li><b>Sequence-to-Sequence Model for Translation:</b> Extension of the model to translation tasks, including a novel pre-training technique and a hierarchical decoding strategy to stabilize latent indicators during generation.</li></ol><p dir="ltr">The study claims enhanced performance in various NLP tasks with results comparable to larger models, with the added benefit of increased interpretability.</p>

Machine learning methods are increasingly being used in the maritime domain to predict traffic anomalies and to mitigate risk, for example avoiding collision and groundingaccidents. However, most machine learning systems used for detecting such issues hasbeen trained predominately on single data sources such as vessel positioning data. Hence,it is desirable to support the means to combine different sources of data - in the trainingphase - to allow more complex models to be built. In this thesis, we propose a multi-data pipeline for accumulating, decoding, preprocessing, and merging Automatic Identification System (AIS) data with weather datato train time series based deep learning models. The pipeline comprises several REST APIsto connect and listen to the data sources, and storing and merging them using StructuredQuery Language (SQL). Specifically, the training pipeline consists of an AIS NMEA message decoder, weather data receiver, and a Postgres database for merging and storing thedata sources. Moreover, the pipeline was assessed by training a TensorFlow vRNN model.The proposed pipeline approach allows flexibility in the inclusion of new data sources toeffectively build models for the maritime domain as well as other traffic domains that usespositioning data.

Time Series Forecasting has become vital in various industries ranging from weather forecasting to business forecasting. There is a need to research database storage solutions for companies in order to optimize resource allocation, enhance decision making process and enable predictive data storage maintenance. With the introduction of Artificial Intelligence and a branch of AI, Machine Learning, Time Series Forecasting has become more powerful and efficient. This project attempts to validate the possibility of using time series forecasting on database storage data to make business predictions. Currently, predicting capabilities of database storage is an area which is not fully explored, despite the growing necessity of databases. Currently, most of the optimization of databases is left to human touch which is ultimately slower and more error prone. As such, this research will investigate the possibilities of time series forecasting in database storage. This project will use Machine Learning and Time-series Forecasting to predict the future trend of database storage to give information on how the trend of the data will change. Examining the pattern of database storage fluctuations will allow the respective owners an overview of their storage and in turn, make decisions on optimizing the database to prevent critical problems ahead of time. Three distinct approaches - employing a traditional linear model fore forecasting, utilizing a Convolutional Neural Network (CNN) to detect local changes in time series data, and leveraging a Recurrent Neural Network (RNN) to capture long term temporal dependencies - are implemented to assess which of these techniques is better suited for the provided dataset. Furthermore, two settings (single step and multi step) have been tested in order to test the changes in accuracy from a small prediction step to a major. The research indicates that currently the models do not have the possibility to be used. This is due to the mean absolute error being very big. The main purpose of the project was to establish which of the three different techniques is the best for the particular dataset provided by the company. In general, across all approaches (Linear, CNN, RNN), their performance was superior in the single step method. In the multi step aspect, The linear model suffered the greatest in the accuracy drop with CNN and RNN performing slightly better. The findings also indicated that the model with local change detection (CNN) performs better for the provided dataset in both single and multi step settings, as evidenced by its minimal Mean Absolute Error (MAE). This is because the dataset is comprised of local data and the models are only trained to check for normal changes. If the research had also checked for seasonality or sequential patterns, then it is possible that LSTM may have had a better outcome due to its capability of capturing those dependencies. The accuracy of single step forecasting using CNN is good (MAE = 0.25) but must be further explored and improved.

This study is exploring the possibilities of automatic morphological analysis of L-verbs in the Palula language by the help from Finite-state technology and two-level morphology along with supervised machine learning. The type of machine learning used are neural Sequence to Sequence models. A morphological transducer is made with the Helsinki Finite-State Transducer Technology, HFST, toolkit covering the L-verbs of the Palula Language. Several Sequence to Sequence models are trained on sets of L-verbs along with morphological tagging annotation. One model is trained with a small amount of manually annotated data and four models are trained with different amounts of training examples generated by the Finite-State Transducer. The efficiency and accuracy of these methods are investigated. The Sequence to Sequence model trained on solely manually annotated data did not perform as well as the other models. A Sequence to Sequence model trained with training examples generated by the transducer performed the best recall, accuracy and F1-score, while the Finite-State Transducer performed the best precision score. / Denna studie undersöker möjligheterna för en automatisk morfologisk analys av L-verb i språket Palula med hjälp av finit tillståndsteknik och två-nivå-morfologi samt övervakad maskininlärning. Den typ av maskininlärning som används i studien är neurala Sekvens till Sekvens-modeller. En morfologisk transduktor är skapad med verktyget Helsinki Finite-State Transducer Technology, HFST, som täcker L-verben i Palula. Flera Sekvens till Sekvens-modeller tränas på set av L-verb med morfologisk taggningsannotation. En modell tränas på ett litet set av manuellt annoterade data och fyra modeller tränas på olika mängder träningsdata som genererats av den finita tillstånds-transduktorn. Effektiviteten och noggrannheten för dessa modeller undersöks. Sekvens till Sekvens-modellen som tränats med bara manuellt annoterade data presterade inte lika bra som de andra modellerna i studien. En Sekvens till Sekvens-modell tränad med träningsdata bestående av genereringar producerade av transduktorn gav bästa svarsfrekvens, noggrannhet och F1-poäng, medan den finita tillstånds-transduktorn gav bästa precision.

In many types of sports, on-screen graphics such as an reconstructed ball trajectory, can be displayed for spectators or players in order to increase understanding. One sub-problem of trajectory reconstruction is tracking of ball positions, which is a difficult problem due to the fast and often complex ball movement. Historically, physics based techniques have been used to track ball positions, but this thesis investigates using a recurrent neural network design, in the application of tracking bouncing golf balls. The network is trained and tested on synthetically created golf ball shots, created to imitate balls shot out from a golf driving range. It is found that the trained network succeeds in tracking golf balls during bounce and roll, with an error rate of under 11 %. / Grafik visad på en skärm, så som en rekonstruerad bollbana, kan användas i många typer av sporter för att öka en åskådares eller spelares förståelse. För att lyckas rekonstruera bollbanor behöver man först lösa delproblemet att följa en bolls positioner. Följning av bollpositioner är ett svårt problem på grund av den snabba och ofta komplexa bollrörelsen. Tidigare har fysikbaserade tekniker använts för att följa bollpositioner, men i den här uppsatsen undersöks en metod baserad på återkopplande neurala nätverk, för att följa en studsande golfbolls bana. Nätverket tränas och testas på syntetiskt skapade golfslag, där bollbanorna är skapade för att imitera golfslag från en driving range. Efter träning lyckades nätverket följa golfbollar under studs och rull med ett fel på under 11 %.

Two mm-wave frequency modulated continuous wave (FMCW) radars were combined with a recurrent neural network (RNN) to perform fall detection. The purpose was to find methods to implement a multi-radar setup for healthcare monitoring and to study the resulting models’ resilience to interference and other obstacles, such as re-arranging the radars in the room. Single-board computers (SBCs) controlled the radars to record and transfer data over Ethernet to a PC. The Ethernet connection also allowed synchronization with the network time protocol (NTP), which was necessary to put the data from the two sensors in correspondence. The proposed RNN used two bidirectional long-short term memory (Bi-LSTM) layers with L2-regularization and dropout layers. It had an overall accuracy of 95.15% and 98.11% recall with a test set. Performance in live testing varied with different arrangements, with an accuracy of 98% with the radars along the same wall, 94% with the radars diagonally, and 90% with an alternative arrangement that the RNN model had not seen during training. However, the latter arrangement resulted in a recall of 95.7%, with false alarms reducing the overall performance. In conclusion, the model performed adequately for fall detection, even with different radar arrangements but could still be sensitive to interference. / Två millimetervågs-radarsystem av typen frequency modulated continuous wave (FMCW) kombinerades för att med hjälp av ett recurrent neural network (RNN) utföra falldetektering. Syftet var att finna metoder för att implementera en multiradarplatform för hälsoövervakning samt att studera de resulterande modellernas tolerans mot interferens och andra hinder så som att radarsystemen placeras på olika sätt i rummet. Enkortsdatorer kontrollerade radarsystemen för att kunna spela in och överföra data över Ethernet till en PC. Ethernetanslutningen möjliggjorde även synkronisering över network time protocol (NTP), vilket var nödvändigt för att sammanlänka datan från de båda sensorerna. Det föreslagna RNN:et använde två dubbelriktade (bidirectional) long-short term memory (Bi-LSTM) lager med L2-regularisering och dropout-lager. Det hade en total noggrannhet på 95.15% och 98.11% recall med ett test-set. Prestandan vid testning i drift varierade beroende på olika uppställningar av radarmodulerna, med en noggrannhet på 98% då de placerades längs samma vägg, 94% då de placerades diagonalt och 90% vid en alternativ uppställning som RNN-modellen inte hade sett när den tränades. Det senare resulterade dock i 95.7% recall, där falsklarm var den främsta felkällan. Sammanfattningsvis presterade modellen bra för falldetektering, även med olika uppställningar, men den verkar fortfarande vara känslig för interferens.

This study aims to investigate the implementation of an AI model that predicts customer purchases, in the telecom industry. The thesis also outlines how such an AI model can assist decision-making in marketing strategies. It is concluded that designing the AI model by following a Recurrent Neural Network (RNN) architecture with a Long Short-Term Memory (LSTM) layer, allow for a successful implementation with satisfactory model performances. Stepwise instructions to construct such model is presented in the methodology section of the study. The RNN-LSTM model further serves as an assisting tool for marketers to assess how a consumer’s website behavior affect their purchase behavior over time, in a quantitative way - by observing what the authors refer to as the Customer Purchase Propensity Journey (CPPJ). The firm empirical basis of CPPJ, can help organizations improve their allocation of marketing resources, as well as benefit the organization’s online presence by allowing for personalization of the customer experience. / Denna studie undersöker implementeringen av en AI-modell som förutspår kunders köp, inom telekombranschen. Studien syftar även till att påvisa hur en sådan AI-modell kan understödja beslutsfattande i marknadsföringsstrategier. Genom att designa AI-modellen med en Recurrent Neural Network (RNN) arkitektur med ett Long Short-Term Memory (LSTM) lager, drar studien slutsatsen att en sådan design möjliggör en framgångsrik implementering med tillfredsställande modellprestation. Instruktioner erhålls stegvis för att konstruera modellen i studiens metodikavsnitt. RNN-LSTM-modellen kan med fördel användas som ett hjälpande verktyg till marknadsförare för att bedöma hur en kunds beteendemönster på en hemsida påverkar deras köpbeteende över tiden, på ett kvantitativt sätt - genom att observera det ramverk som författarna kallar för Kundköpbenägenhetsresan, på engelska Customer Purchase Propensity Journey (CPPJ). Den empiriska grunden av CPPJ kan hjälpa organisationer att förbättra allokeringen av marknadsföringsresurser, samt gynna deras digitala närvaro genom att möjliggöra mer relevant personalisering i kundupplevelsen.

The aim of the study is to apply and compare the performance of two different types of neural networks on the Quick, Draw! dataset and from this determine whether interpreting the sketches as sequences gives a higher accuracy than interpreting them as pixels. The two types of networks constructed were a recurrent neural network (RNN) and a convolutional neural network (CNN). The networks were optimised and the final architectures included five layers. The final evaluation accuracy achieved was 94.2% and 92.3% respectively, leading to the conclusion that the sequential interpretation of the Quick, Draw! dataset is favourable.

Coreference is an important and frequent concept in any form of discourse, and Coreference Resolution (CR) a widely used task in Natural Language Understanding (NLU). In this thesis, we implement and explore two recent models that include the concept of coreference in Recurrent Neural Network (RNN)-based Language Models (LM). Entity and reference decisions are modeled explicitly in these models using attention mechanisms. Both models learn to save the previously observed entities in a set and to decide if the next token created by the LM is a mention of one of the entities in the set, an entity that has not been observed yet, or not an entity. After a theoretical analysis where we compare the two LMs to each other and to a state of the art Coreference Resolution system, we perform an extensive quantitative and qualitative analysis. For this purpose, we train the two models and a classical RNN-LM as the baseline model on the OntoNotes 5.0 corpus with coreference annotation. While we do not reach the baseline in the perplexity metric, we show that the models’ relative performance on entity tokens has the potential to improve when including the explicit entity modeling. We show that the most challenging point in the systems is the decision if the next token is an entity token, while the decision which entity the next token refers to performs comparatively well. Our analysis in the context of a text generation task shows that a wide-spread error source for the mention creation process is the confusion of tokens that refer to related but different entities in the real world, presumably a result of the context-based word representations in the models. Our re-implementation of the DeepMind model by Yang et al. 2016 performs notably better than the re-implementation of the EntityNLM model by Ji et al. 2017 with a perplexity of 107 compared to a perplexity of 131.