This study has investigated whether it is possible to classify time series data originating from a gamified learning management system. By using the school data provided by the gamification company Insert Coin AB, the aim was to distribute the teacher’s supervision more efficiently among students who are more likely to fail. Motivating this is the possibility that the student retention and completion rate can be increased. This was done by using Long short-term memory and convolutional neural networks and Markov chain to classify time series of event data. Since the classes are balanced the classification was evaluated using only the accuracy metric. The results for the neural networks show positive results but overfitting seems to occur strongly for the convolutional network and less so for the Long short-term memory network. The Markov chain show potential but further work is needed to mitigate the problem of a strong correlation between sequence length and likelihood.

No description available.

Photoplethysmography (PPG) is a method to detect blood volume changes in every heartbeat. The peaks in the PPG signal corresponds to the electrical impulses sent by the heart. The duration between each heartbeat varies, and these variances are better known as heart rate variability (HRV). Thus, finding peaks correctly from PPG signals provides the opportunity to measure an accurate HRV. Additional research indicates that deep learning approaches can extract HRV from a PPG signal with significantly greater accuracy compared to other traditional methods. In this study, deep learning classifiers were built to detect peaks in a noise-contaminated PPG signal and to recognize the performed activity during the data recording. The dataset used in this study is provided by the PhysioBank database consisting of synchronized PPG-, acceleration- and gyro data. The models investigated in this study were limited toa one-layer LSTM network with six varying numbers of neurons and four different window sizes. The most accurate model for the peak classification was the model consisting of 256 neurons and a window size of 15 time steps, with a Matthews correlation coefficient (MCC) of 0.74. The model consisted of64 neurons and a window duration of 1.25 seconds resulted in the most accurate activity classification, with an MCC score of 0.63. Concludingly, more optimization of a deep learning approach could lead to promising accuracy on peak detection and thus an accurate measurement of HRV. The probable cause for the low accuracy of the activity classification problem is the limited data used in this study.

With an increasing demand for rapid high-capacity internet, the telecommunication industry is constantly driven to explore and develop new technologies to ensure stable and reliable networks. To provide a competitive internet service in this growing market, proactive detection and prevention of disturbances are key elements for an operator. Therefore, analyzing network traffic for forecasting disturbances is a well-researched area. This study explores the advantages and drawbacks of implementing a long short-term memory model for forecasting codeword errors in a hybrid fiber-coaxial network. Also, the impact of using multivariate and univariate data for training the model is explored. The performance of the long short-term memory model is compared with a multilayer perceptron model. Analysis of the results shows that the long short-term model, in the vast majority of the tests, performs better than the multilayer perceptron model. This result aligns with the hypothesis, that the long short-term memory model’s ability to handle sequential data would be superior to the multilayer perceptron. However, the difference in performance between the models varies significantly based on the characteristics of the used data set. On the set with heavy fluctuations in the sequential data, the long short-term memory model performs on average 44% better. When training the models on data sets with longer sequences of similar values and with less volatile fluctuations, the results are much more alike. The long short-term model still achieves a lower error on most tests, but the difference is never larger than 7%. If a low error is the sole criterion, the long short-term model is the overall superior model. However, in a production environment, factors such as data storage capacity and model complexity should be taken into consideration. When training the models on multivariate and univariate datasets, the results are unambiguous. When training on all three features, ratios of uncorrectable and correctable codewords, and signal-to-noise ratio, the models always perform better. That is, compared to using uncorrectable codewords as the only training data. This aligns with the hypothesis, which is based on the know-how of hybrid fiber-coaxial experts, that correctable codewords and signal-to-noise ratio have an impact on the occurrence of uncorrectable codewords. / På grund av den ökade efterfrågan av högkvalitativt internet, så drivs telekomindustrin till att konsekvent utforska och utveckla nya teknologier som kan säkerställa stabila och pålitliga nätverk. För att kunna erbjuda konkurrenskraftiga internettjänster, måste operatörerna kunna förutse och förhindra störningar i nätverken. Därför är forskningen kring hur man analyserar och förutser störningar i ett nätverk ett väl exploaterat område. Denna studie undersökte för- och nackdelar med att använda en long short-term memory (LSTM) för att förutse kodordsfel i ett hybridfiber-koaxialt nätverk. Utöver detta undersöktes även hur multidimensionell träningsdata påverkade prestandan. I jämförelsesyfte användes en multilayer perceptron (MLP) och dess resultat. Analysen av resultaten visade att LSTM-modellen presterade bättre än MLP-modellen i majoriteten av de utförda testerna. Men skillnaden i prestanda varierade kraftigt, beroende på vilken datauppsättning som användes vid träning och testning av modellerna. Slutsatsen av detta är att i denna studie så är LSTM den bästa modellen, men att det inte går att säga att LSTM presterar bättre på en godtycklig datauppsättning. Båda modellerna presterade bättre när de tränades på multidimensionell data. Vidare forskning krävs för att kunna determinera om LSTM är den mest självklara modellen för att förutse kodordsfel i ett hybridfiber-koaxialt nätverk.

Artificial Intelligence (AI) and Computer Vision (CV) have brought the promise of many applications along with many challenges to solve. The majority of current AI research has been dedicated to single-modal data processing meaning they use only one modality such as visual recognition or text recognition. However, real-world challenges are often a combination of different modalities of data such as text, audio and images. This thesis focuses on solving the Visual Question Answering (VQA) problem which is a significant multi-modal challenge. VQA is defined as a computer vision system that when given a question about an image will answer based on an understanding of both the question and image. The goal is improving the training time of VQA models. In this thesis, Look, Read, Reason and Answer (LoRRA), which is a state-of-the-art architecture, is used as the base model. Then, Reduce Uni-modal Biases (RUBi) is applied to this model to reduce the importance of uni- modal biases in training. Finally, an early stopping strategy is employed to stop the training process once the model accuracy has converged to prevent the model from overfitting. Numerical results are presented which show that training LoRRA with RUBi and early stopping can converge in less than 5 hours. The impact of batch size, learning rate and warm up hyper parameters is also investigated and experimental results are presented. / Graduate

Predictive maintenance plays a crucial role in preventing unexpected equipment failures and maintaining assets in good operating conditions in various systems. One such scenario where predictive maintenance has been widely used is in battery management systems for electronic vehicles based on lithium batteries, where the risk of failure can be reduced by predicting the remaining useful life of the lithium battery. This project developed a DL model based on Long Short-Term Memory networks which was able to generalize new and various kinds of battery. The model was implemented on a low-cost, low-power using embedded artifcial intelligence, which enables local model execution, reducing costs, time, and risks associated with transferring data to the cloud. To further optimize the model and reduce its memory usage, quantization was applied before porting it to an embedded system based on the STM32 MCU. The results show that the model migration was successful, with low memory cost and no signifcant degradation in accuracy. Finally, the memory usage of the prediction model was also analyzed. / Predictiv underhåll har en avgörande roll för att förebygga oväntade utrustningsfel och bibehålla tillgångar i god driftsvillkor i olika system. Ett scenario där predictivt underhåll har använts mycket är i batterihanteringssystem för elfordon baserade på litiumbatterier, där risken för fel kan reduceras genom att förutsäga den återstående användbarhetsperioden för litiumbatteriet. I det här projektet utvecklades djupinlärningsprediktiva modeller med hjälp av Keras sekventiella modell för att representera en ferlagersneural nätverk och en Lång Korttidsminne modell för tidserieprediktion. Dessa modeller implementerades på en lågkostnad, låglägesmikrokontroller med inbyggd artifcial intelligence, vilket möjliggör lokal modellkörning, vilket reducerar kostnader, tid och risker med att överföra data till molnet. För att ytterligare optimera modellen och minska dess minnesfotavtryck tillämpades kvantisering innan den portades till en inbyggd system baserat på STM32 mikrokontroller. Resultaten visar att modellmigrationen var framgångsrik, med låg minneskostnad och ingen signifkant försämring av precisionen. Slutligen analyserades även minnesanvändningen av prediktionsmodellen.

This report assesses different machine learning models’accuracies to predict whether a stock will go up or down invalue in a short term. The models that is used is linear regression,LSTM and Elman RNN. These models was trained on historicalprice data from the Nasdaq Stock Exchange. The idea that thereexist a relationship of the price movement of a stock and its futurevalue is called ’techncial analysis’. The result shows that neitherLSTM nor Elman RNN provides any statistical significance ofits accuracy for any of the implementations. Linear regression,provides a significant accuracy for longer time series predictionof the price when trained on 100 days of data and prediction ofits movement after five more days. / I denna report undersöks olika maskininlärningsmodeller noggrannhet för att förutspå om en aktie kommer att gå upp eller ner i värde på kort sikt. De evaluerade maskininlärningsmodellernamodellerna är följande: linjär regression, LSTM och Elman RNN. Dessa modeller tränades med hjälp av historisk prisdata från Nasdaq Stock Exchange. Ide´en om att det finns ett samband mellan prisrörelsen av en aktie och dess kortsiktiga framtida värde är benämnt som ’teknisk analys’. Resultaten visar att varken LSTM eller Elman RNN förmedlar en noggrannhet med statistisk signifikans för någon av de anänvda implementationerna. Linjär regression förmedlar en statistisk signikant noggrannhet för längre tidserie förutsägelser med träningsdata om 100 dagar och förutsägelse av aktiens rörelse efter fem fler dagar. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm

<p>This research focused on developing adaptable models and protocols for early warning systems for forecasting plant diseases and datasets. It compared the performance of deep learning models in predicting soybean rust disease outbreaks using three years of public epidemiological data and gridded weather data. The models selected were a dense network and a Long Short-Term Memory (LSTM) network. The objectives included evaluating the effectiveness of small citizen science datasets and gridded meteorological weather in sequential forecasting, assessing the ideal window size and important inputs, and exploring the generalizability of the model protocol and models to other diseases. The model protocol was developed using a soybean rust dataset. Both the dense and the LSTM networks produced accuracies of over 90% during optimization. When tested for forecasting, both networks could forecast with an accuracy of 85% or higher over various window sizes. Experiments on window size indicated a minimum input of 8 -11 days. Generalizability was demonstrated by applying the same protocol to a southern corn rust dataset, resulting in 87.8% accuracy. In addition, transfer learning and pre-trained models were tested. Direct transfer learning between disease was not successful, while pre training models resulted both positive and negative results. Preliminary results are reported for building generalizable disease models using epidemiological and weather data that researchers could apply to generate forecasts for new diseases and locations.</p>

In this master thesis, we implement a two-step anomaly detection mechanism for non-recurrent traffic congestions with data collected from public transport buses in Stockholm. We investigate the use of machine learning to model time series data with LSTMs and evaluate the results with a baseline prediction model. The anomaly detection algorithm embodies both collective and contextual expressivity, meaning it is capable of findingcollections of delayed buses and also takes the temporality of the data into account. Results show that the anomaly detection performance benefits from the lower prediction errors produced by the LSTM network. The intersection rule significantly decreases the number of false positives while maintaining the true positive rate at a sufficient level. The performance of the anomaly detection algorithm has been found to depend on the road segment it is applied to, some segments have been identified to be particularly hard whereas other have been identified to be easier than others. The performance of the best performing setup of the anomaly detection mechanism had a true positive rate of 84.3 % and a true negative rate of 96.0 %. / I den här masteruppsatsen implementerar vi en tvåstegsalgoritm för avvikelsedetektering för icke återkommande trafikstockningar. Data är insamlad från kollektivtrafikbussarna i Stockholm. Vi undersöker användningen av maskininlärning för att modellerna tidsseriedata med hjälp av LSTM-nätverk och evaluerar sedan dessa resultat med en grundmodell. Avvikelsedetekteringsalgoritmen inkluderar både kollektiv och kontextuell uttrycksfullhet, vilket innebär att kollektiva förseningar kan hittas och att även temporaliteten hos datan beaktas. Resultaten visar att prestandan hos avvikelsedetekteringen förbättras av mindre prediktionsfel genererade av LSTM-nätverket i jämförelse med grundmodellen. En regel för avvikelser baserad på snittet av två andra regler reducerar märkbart antalet falska positiva medan den höll kvar antalet sanna positiva på en tillräckligt hög nivå. Prestandan hos avvikelsedetekteringsalgoritmen har setts bero av vilken vägsträcka den tillämpas på, där några vägsträckor är svårare medan andra är lättare för avvikelsedetekteringen. Den bästa varianten av algoritmen hittade 84.3 % av alla avvikelser och 96.0 % av all avvikelsefri data blev markerad som normal data.

The stock market is a non-linear field, but many of the best-known portfolio optimization algorithms are based on linear models. In recent years, the rapid development of machine learning has produced flexible models capable of complex pattern recognition. In this paper, we propose two different methods of portfolio optimization; one based on the development of a multivariate time-dependent neural network,thelongshort-termmemory(LSTM),capable of finding lon gshort-term price trends. The other is the linear Markowitz model, where we add an exponential moving average to the input price data to capture underlying trends. The input data to our neural network are daily prices, volumes and market indicators such as the volatility index (VIX).The output variables are the prices predicted for each asset the following day, which are then further processed to produce metrics such as expected returns, volatilities and prediction error to design a portfolio allocation that optimizes a custom utility function like the Sharpe Ratio. The LSTM model produced a portfolio with a return and risk that was close to the actual market conditions for the date in question, but with a high error value, indicating that our LSTM model is insufficient as a sole forecasting tool. However,the ability to predict upward and downward trends was somewhat better than expected and therefore we conclude that multiple neural network can be used as indicators, each responsible for some specific aspect of what is to be analysed, to draw a conclusion from the result. The findings also suggest that the input data should be more thoroughly considered, as the prediction accuracy is enhanced by the choice of variables and the external information used for training. / Aktiemarknaden är en icke-linjär marknad, men många av de mest kända portföljoptimerings algoritmerna är baserad på linjära modeller. Under de senaste åren har den snabba utvecklingen inom maskininlärning skapat flexibla modeller som kan extrahera information ur komplexa mönster. I det här examensarbetet föreslår vi två sätt att optimera en portfölj, ett där ett neuralt nätverk utvecklas med avseende på multivariata tidsserier och ett annat där vi använder den linjära Markowitz modellen, där vi även lägger ett exponentiellt rörligt medelvärde på prisdatan. Ingångsdatan till vårt neurala nätverk är de dagliga slutpriserna, volymerna och marknadsindikatorer som t.ex. volatilitetsindexet VIX. Utgångsvariablerna kommer vara de predikterade priserna för nästa dag, som sedan bearbetas ytterligare för att producera mätvärden såsom förväntad avkastning, volatilitet och Sharpe ratio. LSTM-modellen producerar en portfölj med avkastning och risk som ligger närmre de verkliga marknadsförhållandena, men däremot gav resultatet ett högt felvärde och det visar att vår LSTM-modell är otillräckligt för att använda som ensamt predikteringssverktyg. Med det sagt så gav det ändå en bättre prediktion när det gäller trender än vad vi antog den skulle göra. Vår slutsats är därför att man bör använda flera neurala nätverk som indikatorer, där var och en är ansvarig för någon specifikt aspekt man vill analysera, och baserat på dessa dra en slutsats. Vårt resultat tyder också på att inmatningsdatan bör övervägas mera noggrant, eftersom predikteringsnoggrannheten.