Providers of online services have witnessed a rapid growth of their user base in the last few years. The phenomenon has attracted an increasing number of competitors determined on obtaining their own share of the market. In this context, the cost of attracting new customers has increased significantly, raising the importance of retaining existing clients. Therefore, it has become progressively more important for the companies to improve user experience and ensure they keep a larger share of their users active in consuming their product. Companies are thus compelled to build tools that can identify what prompts customers to stay and also identify the users intent on abandoning the service. The focus of this thesis is to address the problem of predicting user abandonment, also known as "churn", and also detecting motives for user retention on data provided by an online streaming service. Classical models like logistic regression and random forests have been used to predict the churn probability of a customer with a fair amount of precision in the past, commonly by aggregating all known information about a user over a time period into a unique data point. On the other hand, recurrent neural networks, especially the long short-term memory (LSTM) variant, have shown impressive results for other domains like speech recognition and video classification, where the data is treated as a sequence instead. This thesis investigates how LSTM models perform for the task of predicting churn compared to standard nonsequential baseline methods when applied to user behavior data of a music streaming service. It was also explored how different aspects of the data, like the distribution between the churning and retaining classes, the size of user event history and feature representation influences the performance of predictive models. The obtained results show that LSTMs has a comparable performance to random forest for churn detection, while being significantly better than logistic regression.  Additionally, a framework for creating a dataset suitable for training predictive models is provided, which can be further explored as to analyze user behavior and to create retention actions that minimize customer abandonment. / Leverantörer av onlinetjänster har bevittnat en snabb användartillväxt under de senaste åren. Denna trend har lockat ett ökande antal konkurrenter som vill ta del av denna växande marknad. Detta har resulterat i att kostnaden för att locka nya kunder ökat avsevärt, vilket även ökat vikten av att behålla befintliga kunder. Det har därför gradvis blivit viktigare för företag att förbättra användarupplevelsen och se till att de behåller en större andel avanvändarna aktiva. Företag har därför ett starkt intresse avatt bygga verktyg som kan identifiera vad som driver kunder att stanna eller vad som får dem lämna. Detta arbete fokuserar därför på hur man kan prediktera att en användare är på väg att överge en tjänst, så kallad “churn”, samt identifiera vad som driver detta baserat på data från en onlinetjänst.   Klassiska modeller som logistisk regression och random forests har tidigare använts på aggregerad användarinformation över en given tidsperiod för att med relativt god precision prediktera sannolikheten för att en användare kommer överge produkten.  Under de senaste åren har dock sekventiella neurala nätverk (särskilt LSTM-varianten Long Short Term Memory), där data istället behandlas som sekvenser, visat imponerande resultat för andra domäner såsom taligenkänning och videoklassificering. Detta arbete undersöker hur väl LSTM-modeller kan användas för att prediktera churn jämfört med traditionella icke-sekventiella metoder när de tillämpas på data över användarbeteende från en musikstreamingtjänst. Arbetet undersöker även  hur olika aspekter av data påverkar prestandan av modellerna inklusive distributionen mellan gruppen av användare som överger produkten mot de som stannar, längden av användarhändelseshistorik och olika val av användarfunktioner för modeller och användardatan. De erhållna resultaten visar att LSTM har en jämförbar prestanda med random forest för prediktering av användarchurn  samt är signifikant bättre än logistisk regression. LSTMs visar sig således vara ett lämpligt val för att förutsäga churn på användarnivå. Utöver dessa resultat utvecklades även ett ramverk  för att skapa dataset som är lämpliga för träning av prediktiva modeller, vilket kan utforskas ytterligare för att analysera användarbeteende och för att skapa förbättrade åtgärder för att behålla användare och minimera antalet kunder som överger tjänsten.

Neural network approaches have become popular in the field of automatic speech recognition (ASR). Most ASR methods use audio data to classify words. Lip reading ASR techniques utilize only video data, which compensates for noisy environments where audio may be compromised. A comprehensive approach, including the vetting of datasets and development of a preprocessing chain, to video-based ASR is developed. This approach will be based on neural networks, namely 3D convolutional neural networks (3D-CNN) and Long short-term memory (LSTM). These types of neural networks are designed to take in temporal data such as videos. Various combinations of different neural network architecture and preprocessing techniques are explored. The best performing neural network architecture, a CNN with bidirectional LSTM, compares favorably against recent works on video-based ASR.

As Next Generation (NextG) networks become more complex, the need to develop a robust, reliable network traffic prediction framework for intelligent network management increases. This study compares the performance of machine learning models in network traffic prediction using a custom Service-Level Agreement (SLA) - based loss function to ensure SLA violation constraints while minimizing overprovisioning. The proposed SLA-based parametric custom loss functions are used to maintain the SLA violation rate percentages the network operators require. Our approach is multivariate, spatiotemporal, and SLA-driven, incorporating 20 Radio Access Network (RAN) features, custom peak traffic time features, and custom mobility-based clustering to leverage spatiotemporal relationships. In this study, five machine learning models are considered: one recurrent neural network (LSTM) model, two encoder-decoder architectures (Transformer and Autoformer), and two gradient-boosted tree models (XGBoost and LightGBM). The prediction performance of the models is evaluated based on different metrics such as SLA violation rate constraints, overprovisioning, and the custom SLA-based loss function parameter. According to our evaluations, Transformer models with custom peak time features achieve the minimum overprovisioning volume at 3% SLA violation constraint. Gradient-boosted tree models have lower overprovisioning volumes at higher SLA violation rates. / Master of Science / As the Next Generation (NextG) networks become more complex, the need to develop a robust, reliable network traffic prediction framework for intelligent network management increases. This study compares the performance of machine learning models in network traffic prediction using a custom loss function to ensure SLA violation constraints. The proposed SLA-based custom loss functions are used to maintain the SLA violation rate percentages required by the network operators while minimizing overprovisioning. Our approach is multivariate, spatiotemporal, and SLA-driven, incorporating 20 Radio Access Network (RAN) features, custom peak traffic time features, and mobility-based clustering to leverage spatiotemporal relationships. We use five machine learning and deep learning models for our comparative study: one recurrent neural network (RNN) model, two encoder-decoder architectures, and two gradient-boosted tree models. The prediction performance of the models was evaluated based on different metrics such as SLA violation rate constraints, overprovisioning, and the custom SLA-based loss function parameter.

No description available.

Building energy load forecasting is becoming an increasingly important task with the rapid deployment of smart homes, integration of renewables into the grid and the advent of decentralized energy systems. Residential load forecasting has been a challenging task since the residential load is highly stochastic. Deep learning models have showed tremendous promise in the fields of time-series and sequential data and have been successfully used in the field of short-term load forecasting at the building level. Although, other studies have looked at using deep learning models for building energy forecasting, most of those studies have looked at limited number of homes or an aggregate load of a collection of homes. This study aims to address this gap and serve as an investigation on selecting the better deep learning model architecture for short term load forecasting on 3 communities of residential buildings. The deep learning models CNN and LSTM have been used in the study. For 15-min ahead forecasting for a collection of homes it was found that homes with a higher variance were better predicted by using CNN models and LSTM showed better performance for homes with lower variances. The effect of adding weather variables on 24-hour ahead forecasting was studied and it was observed that adding weather parameters did not show an improvement in forecasting performance. In all the homes, deep learning models are shown to outperform the simple ANN model. / Master of Science / Building energy load forecasting is becoming an increasingly important task with the rapid deployment of smart homes, integration of renewables into the grid and the advent of decentralized energy systems. Residential load forecasting has been a challenging task since residential load is highly stochastic. Deep learning models have showed tremendous promise in the fields of time-series and sequential data and have been successfully used in the field of short-term load forecasting. Although, other studies have looked at using deep learning models for building energy forecasting, most of those studies have looked at only a single home or an aggregate load of a collection of homes. This study aims to address this gap and serve as an analysis on short term load forecasting on 3 communities of residential buildings. Detailed analysis on the model performances across all homes have been studied. Deep learning models have been used in this study and their efficacy is measured compared to a simple ANN model.

Yes / People start posting tweets containing texts, images, and videos as soon as a disaster hits an area. The analysis of these disaster-related tweet texts, images, and videos can help humanitarian response organizations in better decision-making and prioritizing their tasks. Finding the informative contents which can help in decision making out of the massive volume of Twitter content is a difficult task and require a system to filter out the informative contents. In this paper, we present a multi-modal approach to identify disaster-related informative content from the Twitter streams using text and images together. Our approach is based on long-short-term-memory (LSTM) and VGG-16 networks that show significant improvement in the performance, as evident from the validation result on seven different disaster-related datasets. The range of F1-score varied from 0.74 to 0.93 when tweet texts and images used together, whereas, in the case of only tweet text, it varies from 0.61 to 0.92. From this result, it is evident that the proposed multi-modal system is performing significantly well in identifying disaster-related informative social media contents.

In this paper we explore various approaches to using deep neural networks to per- form cryptanalysis, with the ultimate goal of having a deep neural network deci- pher encrypted data. We use long short-term memory networks to try to decipher encrypted text and we use a convolutional neural network to perform classification tasks on encrypted MNIST images. We find that although the network is unable to decipher encrypted data, it is able to perform classification on encrypted data. We also find that the networks performance is depending on what key were used to en- crypt the data. These findings could be valuable for further research into the topic of cryptanalysis using deep neural networks.

The study of human locomotion, known as gait analysis, has for a long time been performed withexpensive equipment in laboratory settings. However, the emergence of machine learning sparkedinterest in integrating this technology in gait analysis, thus simplifying the process. This study’saim is to substitute the pressure insoles used during gait cycle analysis of a walking subject, with amachine learning model.To achieve this, a model based on Long-Short Term Memory networks that predicts vertical groundreaction force based on data from inertial measurement unit sensors was used. This serves as asubstitution for pressure insoles or pressure plates. The model was trained with time series datasetscontaining inertial measurement unit data and corresponding pressure insole data. Subsequently, itwas tested for intersubjective, out-of-sample data.The model was able to capture the periodicity of the gait cycle as well as predict the general shapeof the vertical ground reaction force curves, where the accuracy was quantified using normalisedroot mean squared error. The error was in a range between 17.8% and 13.4% and had an average of15.2%, when tested intersubjectively and out-of-sample. The most significant factor contributing tothe error was the model’s amplitude inaccuracies which was, most likely, due to information beinglost during the processing of the data, as well as simply having an insufficient amount of data.

Den nya generationens digitalisering har slagit rot i samhället. Algoritmer och datamodeller styr nyhetsflödet i social media, röststyr mobilen genom att tolka rösten och självstyr bilen, helt och hållet i autonoma fordon. Inom industrierna finns det också en pågående process där machine learning kan appliceras för att öka drifttillgänglighet och minska kostnader. Det nuvarande paradigmet för att underhålla icke-säkerhetsklassade maskiner i kärnkraftindustrin är en kombination av Avhjälpande Underhåll och Förebyggande Underhåll. Avhjälpande underhåll innebär att underhålla maskinen när fel inträffar, förebyggande underhåll innebär att underhålla med periodiska intervall. Båda sätten är kostsamma för att de riskerar att under- respektive över-underhålla maskinen och blir därmed resurskrävande. Ett paradigmskifte är på väg, det stavas Prediktivt Underhåll - att kunna förutspå fel innan de inträffar och  planera underhåll därefter. Den här rapporten utforskar möjligheten att använda sig av de neurala nätverken LSTM och GRU för att kunna prognostisera eventuella skador på maskiner. Det här baseras på mätdata och historiska fel på maskinen. / The new generation of digitalization has been ingrained into society. Algorithms and data models are controlling the news feed of social media, controlling the phone by interpreting voices and controlling the car, altogether with automonous vehicles. In the industries there is also an ongoing process where machine learning is applied to increase availability and reduce costs. The current paradigm for maintaining non-critical machines in the nuclear power industry is a combination of corrective maintenance and preventive maintenance. Corrective maintenance means doing repairs on the machine upon faults, preventive maintenance means doing repairs periodically. Both ways are costly because they run the risk of under- and over-maintaining the machine and therefore becoming resource-intensive. A paradigm shift is on it's way, and it's spelled Predictive Maintenance - being able to predict faults before they happen and plan maintenance thence. This report explores the possibilities of using LSTM and GRU to forecast potential damage on machines. This is based on data from measurements and historical issues on the machine.

I detta arbete jämfördes ett LSTM-nätverk med ett feedforward-nätverk för generellt Atari 2600 spelande. Prestandan definierades som poängen agenten får för ett visst spel. Hypotesen var att LSTM skulle prestera minst lika bra som feedforward och förhoppningsvis mycket bättre. För att svara på frågeställningen skapades två olika agenter, en med ett LSTM-nätverk och en med ett feedforward-nätverk. Experimenten utfördes på Stella emulatorn med hjälp av ramverket the Arcade Learning Environment (ALE). Hänsyn togs till Machado råd om inställningar för användning av ALE och hur agenter borde tränas och evalueras samtidigt. Agenterna utvecklades med hjälp av en genetisk algoritm. Resultaten visade att LSTM var minst lika bra som feedforward men båda metoderna blev slagna av Machados metoder. Toppoängen i varje spel jämfördes med Granfelts arbete som har varit en utgångspunkt för detta arbete.