Customer churn is problematic for any business trying to expand their customer base. The acquisition of new customers to replace churned ones are associated with additional costs, whereas taking measures to retain existing customers may prove more cost efficient. As such, it is of interest to estimate the time until the occurrence of a potential churn for every customer in order to take preventive measures. The application of deep learning and machine learning to this type of problem using time series data is relatively new and there is a lot of recent research on this topic. This thesis is based on the assumption that early signs of churn can be detected by the temporal changes in customer behavior. Recurrent neural networks and more specifically long short-term memory (LSTM) and gated recurrent unit (GRU) are suitable contenders since they are designed to take the sequential time aspect of the data into account. Random forest (RF) and stochastic vector machine (SVM) are machine learning models that are frequently used in related research. The problem is solved through a classification approach, and a comparison is done with implementations using LSTM, GRU, RF, and SVM. According to the results, LSTM and GRU perform similarly while being slightly better than RF and SVM in the task of predicting customers that will churn in the coming six months, and that all models could potentially lead to cost savings according to simulations (using non-official but reasonable costs assigned to each prediction outcome). Predicting the time until churn is a more difficult problem and none of the models can give reliable estimates, but all models are significantly better than random predictions. / Kundbortfall är problematiskt för företag som försöker expandera sin kundbas. Förvärvandet av nya kunder för att ersätta förlorade kunder är associerat med extra kostnader, medan vidtagandet av åtgärder för att behålla kunder kan visa sig mer lönsamt. Som så är det av intresse att för varje kund ha pålitliga tidsestimat till en potentiell uppsägning kan tänkas inträffa så att förebyggande åtgärder kan vidtas. Applicerandet av djupinlärning och maskininlärning på denna typ av problem som involverar tidsseriedata är relativt nytt och det finns mycket ny forskning kring ämnet. Denna uppsats är baserad på antagandet att tidiga tecken på kundbortfall kan upptäckas genom kunders användarmönster över tid. Reccurent neural networks och mer specifikt long short-term memory (LSTM) och gated recurrent unit (GRU) är lämpliga modellval eftersom de är designade att ta hänsyn till den sekventiella tidsaspekten i tidsseriedata. Random forest (RF) och stochastic vector machine (SVM) är maskininlärningsmodeller som ofta används i relaterad forskning. Problemet löses genom en klassificeringsapproach, och en jämförelse utförs med implementationer av LSTM, GRU, RF och SVM. Resultaten visar att LSTM och GRU presterar likvärdigt samtidigt som de presterar bättre än RF och SVM på problemet om att förutspå kunder som kommer att säga upp sig inom det kommande halvåret, och att samtliga modeller potentiellt kan leda till kostnadsbesparingar enligt simuleringar (som använder icke-officiella men rimliga kostnader associerat till varje utfall). Att förutspå tid till en kunduppsägning är ett svårare problem och ingen av de framtagna modellerna kan ge pålitliga tidsestimat, men alla är signifikant bättre än slumpvisa gissningar.

Predicting the next location of a user has been interesting for both academia and industry. Applications like location-based advertising, traffic planning, intelligent resource allocation as well as in recommendation services are some of the problems that many are interested in solving. Along with the technological advancement and the widespread usage of electronic devices, many location-based records are created. Today, deep learning framework has successfully surpassed many conventional methods in many learning tasks, most known in the areas of image and voice recognition. One of the neural network architecture that has shown the promising result at sequential data is Recurrent Neural Network (RNN). Since the creation of RNN, much alternative architecture have been proposed, and architectures like Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU) are one of the popular ones that are created[5]. This thesis uses GRU architecture and features that incorporate time and location into the network to forecast people’s next location In this paper, a spatial-temporal neural network (ST-GRU) has been proposed. It can be seen as two parts, which are ST and GRU. The first part is a feature extraction algorithm that pulls out the information from a trajectory into location sequences. That process transforms the trajectory into a friendly sequence format in order to feed into the model. The second part, GRU is proposed to predict the next location given a user’s trajectory. The study shows that the proposed model ST-GRU has the best results comparing the baseline models. / Att förutspå vart en individ är på väg har varit intressant för både akademin och industrin. Tillämpningar såsom platsbaserad annonsering, trafikplanering, intelligent resursallokering samt rekommendationstjänster är några av de problem som många är intresserade av att lösa. Tillsammans med den tekniska utvecklingen och den omfattande användningen av elektroniska enheter har många platsbaserade data skapats. Idag har tekniken djupinlärning framgångsrikt överträffat många konventionella metoder i inlärningsuppgifter, bland annat inom områdena bild och röstigenkänning. En neural nätverksarkitektur som har visat lovande resultat med sekventiella data kallas återkommande neurala nätverk (RNN). Sedan skapandet av RNN har många alternativa arkitekturer skapats, bland de mest kända är Long Short Term Memory (LSTM) och Gated Recurrent Units (GRU). Den här studien använder en modifierad GRU där man bland annat lägger till attribut såsom tid och distans i nätverket för att prognostisera nästa plats. I det här examensarbetet har ett rumsligt temporalt neuralt nätverk (ST-GRU) föreslagits. Den består av två delar, nämligen ST och GRU. Den första delen är en extraktionsalgoritm som drar ut relevanta korrelationer mellan tid och plats som är inkorporerade i nätverket. Den andra delen, GRU, förutspår nästa plats med avseende på användarens aktuella plats. Studien visar att den föreslagna modellen ST-GRU ger bättre resultat jämfört med benchmarkmodellerna.

Synthetic short positions constructed by equity options and stock loan short sells are linked by arbitrage. This thesis analyses the link by considering the implied volatility surface (IVS) at 80%, 100%, and 120% moneyness, and stock loan variables such as benchmark rate (rt), utilization, short interest, and transaction trends to inspect time-dependent structures between the two assets. By applying multiple multivariate time-series analyses in terms of vector autoregression (VAR) and the recurrent neural networks long short-term memory (LSTM) and gated recurrent units (GRU) with a sliding window methodology. This thesis discovers linear and complex relationships between the IVS and stock loan data. The three-day-ahead out-of-sample LSTM forecast of IV at 80% moneyness improved by including lagged values of rt and yielded 19.6% MAPE and forecasted correct direction 81.1% of samples. The corresponding 100% moneyness GRU forecast was also improved by including stock loan data, at 10.8% MAPE and correct directions for 60.0% of samples. The 120% moneyness VAR forecast did not improve with stock loan data at 26.5% MAPE and correct directions for 66.2% samples. The one-month-ahead rt VAR forecast improved by including a lagged IVS, at 25.5% MAPE and 63.6% correct directions. The presented data was optimal for each target variable, showing that the application of LSTM and GRU was justified. These results indicate that considering stock loan data when forecasting IVS for 80% and 100% moneyness is advised to gain exploitable insights for short-term positions. They are further validated since the different models yielded parallel inferences. Similar analysis with other equity is advised to gain insights into the relationship and improve such forecasts. / Syntetiska kortpositioner konstruerade av aktieoptioner och blankning med aktielån är kopplade med arbitrage. Denna tes analyserar kopplingen genom att överväga den implicerade volatilitetsytan vid 80%, 100% och 120% moneyness och aktielånvariabler såsom referensränta rt, låneutnyttjande, låneintresse, och transaktionstrender för att granska tidsberoende strukturer mellan de två tillgångarna. Genom att tillämpa multipel multidimensionell tidsserieanalys såsom vektorautoregression (VAR) och de rekursiva neurala nätverken long short-term memory (LSTM) och gated recurrent units (GRU). Tesen upptäcker linjära och komplexa samband mellan implicerade volatilitetsytor och aktielånedata. Tre dagars LSTM-prognos av implicerade volatiliteten vid 80% moneyness förbättrades genom att inkludera fördröjda värden av rt och gav 19,6% MAPE och prognostiserade korrekt riktning för 81,1% av prover. Motsvarande 100% moneyness GRU-prognos förbättrades också genom att inkludera aktielånedata, resulterande i 10,8% MAPE och korrekt riktning för 60,0% av prover. VAR-prognosen för 120% moneyness förbättrades inte med alternativa data på 26,5% MAPE och korrekt riktning för 66,2% av prover. En månads VAR-prognos för rt förbättrades genom att inkludera en fördröjd implicerad volatilitetsyta, resulterande i 25,5% MAPE och 63,6% korrekta riktningar. Presenterad statistik var optimala för dessa variabler, vilket visar att tillämpningen av LSTM och GRU var motiverad. Därav rekommenderas det att inkludera aktielånedata för prognostisering av implicerade volatilitetsytor för 80% och 100% moneyness, speciellt för kortsiktiga positioner. Resultaten valideras ytterligare eftersom de olika modellerna gav dylika slutsatser. Liknande analys med andra aktier är rekommenderat för att få insikter i förhållandet och förbättra sådana prognoser.

As modern systems are becoming increasingly complex, they are also becoming more and more cumbersome to diagnose and fix when things go wrong. One domain where it is very important for machinery and equipment to stay functional is in the world of medical IT, where technology is used to improve healthcare for people all over the world. This thesis aims to help with reducing downtime on critical life-saving equipment by implementing automatic analysis of system logs that without any domain experts involved can give an indication of the state that the system is in. First, a literature study was performed where three potential candidates of suitable neural network architectures was found. Next, the networks were implemented and a data pipeline for collecting and labeling training data was set up. After training the networks and testing them on a separate data set, the best performing model out of the three was based on GRU (Gated Recurrent Unit). Lastly, this model was tested on some real world system logs from two different sites, one without known issues and one with slow image import due to network issues. The results showed that it was feasible to build such a system that can give indications on external parameters such as network speed, latency and packet loss percentage using only raw system logs as input data. GRU, 1D-CNN (1-Dimensional Convolutional Neural Network) and Transformer's Encoder are the three models that were tested, and the best performing model was shown to produce correct patterns even on the real world system logs. / I takt med att moderna system ökar i komplexitet så blir de även svårare att felsöka och reparera när det uppstår problem. Ett område där det är mycket viktigt att maskiner och utrustning fungerar korrekt är inom medicinsk IT, där teknik används för att förbättra hälso- och sjukvården för människor över hela världen. Syftet med denna avhandling är att bidra till att minska tiden som kritisk livräddande utrustning inte fungerar genom att implementera automatisk analys av systemloggarna som utan hjälp av experter inom området kan ge en indikation på vilket tillstånd som systemet befinner sig i. Först genomfördes en litteraturstudie där tre lovande typer av neurala nätverk valdes ut. Sedan implementerades dessa nätverk och det sattes upp en datapipeline för insamling och märkning av träningsdata. Efter att ha tränat nätverken och testat dem på en separat datamängd så visade det sig att den bäst presterande modellen av de tre var baserad på GRU (Gated Recurrent Unit). Slutligen testades denna modell på riktiga systemloggar från två olika sjukhus, ett utan kända problem och ett där bilder importerades långsamt på grund av nätverksproblem. Resultaten visade på att det är möjligt att konstruera ett system som kan ge indikationer på externa parametrar såsom nätverkshastighet, latens och paketförlust i procent genom att enbart använda systemloggar som indata.  De tre modeller som testades var GRU, 1D-CNN (1-Dimensional Convolutional Neural Network) och Transformer's Encoder. Den bäst presterande modellen visade sig kunna producera korrekta mönster även för loggdata från verkliga system.

In this thesis we forecast the future signal strength of base stations in mobile networks. Better forecasts might improve handover of mobile phones between base stations, thus improving overall user experience. Future values are forecast using a series of past sig- nal strength measurements. We use vector autoregression (VAR), a multilayer perceptron (MLP), and a gated recurrent unit (GRU) network. Hyperparameters, including the set of lags, of these models are optimised using Bayesian optimisation (BO) with Gaussian pro- cess (GP) priors. In addition to BO of the VAR model, we optimise the set of lags in it using a standard bottom-up and top-down heuristic. Both approaches result in similar predictive mean squared error (MSE) for the VAR model, but BO requires fewer model estimations. The GRU model provides the best predictive performance out of the three models. How- ever, none of the models (VAR, MLP, or GRU) achieves the accuracy required for practical applicability of the results. Therefore, we suggest adding more information to the model or reformulating the problem.

The success of Internet of Things solutions allowed the establishment of new applications such as smart hydroponic agriculture. One typical problem in such an application is the rapid degradation of the deployed sensors. Traditionally, this problem is resolved by frequent manual maintenance, which is considered to be ineffective and may harm the crops in the long run. The main purpose of this thesis was to propose a machine learning approach for automating the detection of sensor fault drifts. In addition, the solution’s operability was investigated in a cloud computing environment in terms of the response time. This thesis proposes a detection algorithm that utilizes RNN in predicting sensor drifts from time-series data streams. The detection algorithm was later named; Predictive Sliding Detection Window (PSDW) and consisted of both forecasting and classification models. Three different RNN algorithms, i.e., LSTM, CNN-LSTM, and GRU, were designed to predict sensor drifts using forecasting and classification techniques. The algorithms were compared against each other in terms of relevant accuracy metrics for forecasting and classification. The operability of the solution was investigated by developing a web server that hosted the PSDW algorithm on an AWS computing instance. The resulting forecasting and classification algorithms were able to make reasonably accurate predictions for this particular scenario. More specifically, the forecasting algorithms acquired relatively low RMSE values as ~0.6, while the classification algorithms obtained an average F1-score and accuracy of ~80% but with a high standard deviation. However, the response time was ~5700% slower during the simulation of the HTTP requests. The obtained results suggest the need for future investigations to improve the accuracy of the models and experiment with other computing paradigms for more reliable deployments.

Deep learning models have been used widely in various areas and applications of our everyday lives. They could also change the way non-profit organizations work and help optimize fundraising results. In this thesis, sequential models are applied in fundraising to compare their performance against the traditional machine learning model. Sequential model is a type of neural network that is specialized for processing sequential data. Although some research utilizing machine learning algorithms in fundraising context exists, it is based on the data extracted from the specific time window, which does not take time-dependency of features into account; therefore, time-series features are independent at each data point relative to others. This approach results in loss of time notion. In this thesis, we experiment with the application of time-dependent sequential models including Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU) and their variants in the fundraising domain to predict the alumni monetary contribution to the university. We also expand our study by including the architecture that treats time-invariant demographic data as a condition to the sequential layers. In this model, the time-dependent data is concatenated after running the sequential model. Sequential deep learning is empirically evaluated and compared against the traditional machine learning models. The results demonstrate the potential use of both traditional machine learning and sequential deep learning in the prediction of fundraising outcomes and offer non-profit organizations solutions to achieve their mission. / Graduate

Language modeling is a very broad field and has been used for various purposes for a long period of time to make the lives of people easier. Language modeling is also used for text prediction for mobile keyboards to make the user experience smooth. Tobii has been working since 2001 for users who are suffering from ALS (Amyotrophic Lateral Sclerosis). In this disease, users are unable to talk, walk or chew due to the weakening of voluntary muscles and this gets worse day by day. Tobii has designed an Eye Tracker solution for people suffering from ALS to do their tasks more conveniently. They also developed a keyboard for talking which is controlled by an Eye Tracker device. Users can write sentences using the keyboard and then convey them to other people by conversion of this keyboard written text to speech. Therefore, the thesis is related to predicting the text on the initial input of the keyboard to make the user experience fast, easy and less hectic. This thesis project was conducted at Tobii Dynavox with the objective to build a language model which is an automatic, fast, and efficient approach to predict the text for the given input of text. It explores the way to predict sentences by using deep learning models on the initial text input from users and predict the text by taking into consideration user-specific writing style. The model developed in the thesis could be used by Tobii Dynavox for the end-users to predict the text. Part of the objective is also to find out which is the better approach for the implementation of the language models. The results show that federated learning is performing better than centralized machine learning. After analysing the results, it can also be said that Gated Recurrent Units (GRU) will be a good choice for our models because these models show better results for accuracy and take less training and response times.

The previous research available to predict travel speed is wide and has been extensively studied. What currently is missing from the previous work is to estimate the travel speed when different non-recurrent events occur, such as car accidents and road maintenance work. This research implements a machine learning model to predict the average speed on a road segment with and without road accidents. The model would assist in (1) planning the most efficient route which could reduce CO2 emissions and travel time (2) the drivers in traffic could get an estimate of when the traffic will open up again (3) the authorities could take safety measures if drivers are expected to be stuck for too long. In our work, we conducted a review to determine some of the optimal machine learning models to predict on time series data. What we found by comparing GRU (Gated Recurrent Unit) and LSTM (Long Short Term Memory) on travel speed data over a road in Sweden provided by the Swedish Transport Administration, is that there is no major difference in performance between the LSTM and GRU algorithms to predict the average travel speed. We also study the impact of using weather, date and accident related parameters on the model’s predictions. What we found is that we obtained much better results when including the weather data. Furthermore, the inclusion of road events vaguely hints that it could improve performance, but can not be verified due to the low number of road accidents in our dataset.

Pretendemos com este trabalho contribuir para evidenciar a atualidade do debate conceitual sobre a educação popular, em diálogo com o fazer artístico e a intencionalidade política. Para tanto, lançamos mão da observação participante junto a uma iniciativa de grupos de teatro de grupo da cidade de São Paulo, de criação de espaços coletivos de ação e reflexão sobre seu fazer artístico, investigando potencialidades e limites de sua contribuição estética, políti-ca e pedagógica para a formação cultural e política - e conscientização - da classe trabalhado-ra. Nessa trajetória emergiram questões fundamentais para a construção de uma atuação transformadora do ponto de vista estético, das condições materiais, do modo de produção, das formas organizativas dos trabalhadores e trabalhadoras da arte e do teatro, da relação com o público, da interação com movimentos sociais. Tais questionamentos nos remeteram a sua historicidade, identificando continuidades, descontinuidades, contradições e supera-ções em relação a experiências anteriores em que a arte, assim como a educação, como as-pectos da cultura estiveram imbricadas em uma estratégia política de transformação social. / We intend with this work contribute to highlighting the relevance of the conceptual debate on popular education, in dialogue with artistic practice and the political intentionality. For this, we used participant observation with an initiative of theater groups \"group\" of São Pau-lo, of creating collective spaces of action and reflection on their artistic practice, investigating the potential and limits of its aesthetic , political and educational contribution to the cultural and political - and raising awareness - of the working class. Along the way key issues emerged for the construction of a transforming performance - from the aesthetic point of view, the material conditions, of the production process, the organizational forms of workers of art and theater, the relationship with the public, the interaction with social movements. Such questions have referred to their historicity, identifying continuities, discontinuities, contra-dictions and overcomes from previous experiences when art, as well as education, as cultur-al aspects, were intertwined in a political strategy of social transformation.