In this master thesis, we implement a two-step anomaly detection mechanism for non-recurrent traffic congestions with data collected from public transport buses in Stockholm. We investigate the use of machine learning to model time series data with LSTMs and evaluate the results with a baseline prediction model. The anomaly detection algorithm embodies both collective and contextual expressivity, meaning it is capable of findingcollections of delayed buses and also takes the temporality of the data into account. Results show that the anomaly detection performance benefits from the lower prediction errors produced by the LSTM network. The intersection rule significantly decreases the number of false positives while maintaining the true positive rate at a sufficient level. The performance of the anomaly detection algorithm has been found to depend on the road segment it is applied to, some segments have been identified to be particularly hard whereas other have been identified to be easier than others. The performance of the best performing setup of the anomaly detection mechanism had a true positive rate of 84.3 % and a true negative rate of 96.0 %. / I den här masteruppsatsen implementerar vi en tvåstegsalgoritm för avvikelsedetektering för icke återkommande trafikstockningar. Data är insamlad från kollektivtrafikbussarna i Stockholm. Vi undersöker användningen av maskininlärning för att modellerna tidsseriedata med hjälp av LSTM-nätverk och evaluerar sedan dessa resultat med en grundmodell. Avvikelsedetekteringsalgoritmen inkluderar både kollektiv och kontextuell uttrycksfullhet, vilket innebär att kollektiva förseningar kan hittas och att även temporaliteten hos datan beaktas. Resultaten visar att prestandan hos avvikelsedetekteringen förbättras av mindre prediktionsfel genererade av LSTM-nätverket i jämförelse med grundmodellen. En regel för avvikelser baserad på snittet av två andra regler reducerar märkbart antalet falska positiva medan den höll kvar antalet sanna positiva på en tillräckligt hög nivå. Prestandan hos avvikelsedetekteringsalgoritmen har setts bero av vilken vägsträcka den tillämpas på, där några vägsträckor är svårare medan andra är lättare för avvikelsedetekteringen. Den bästa varianten av algoritmen hittade 84.3 % av alla avvikelser och 96.0 % av all avvikelsefri data blev markerad som normal data.

The stock market is a non-linear field, but many of the best-known portfolio optimization algorithms are based on linear models. In recent years, the rapid development of machine learning has produced flexible models capable of complex pattern recognition. In this paper, we propose two different methods of portfolio optimization; one based on the development of a multivariate time-dependent neural network,thelongshort-termmemory(LSTM),capable of finding lon gshort-term price trends. The other is the linear Markowitz model, where we add an exponential moving average to the input price data to capture underlying trends. The input data to our neural network are daily prices, volumes and market indicators such as the volatility index (VIX).The output variables are the prices predicted for each asset the following day, which are then further processed to produce metrics such as expected returns, volatilities and prediction error to design a portfolio allocation that optimizes a custom utility function like the Sharpe Ratio. The LSTM model produced a portfolio with a return and risk that was close to the actual market conditions for the date in question, but with a high error value, indicating that our LSTM model is insufficient as a sole forecasting tool. However,the ability to predict upward and downward trends was somewhat better than expected and therefore we conclude that multiple neural network can be used as indicators, each responsible for some specific aspect of what is to be analysed, to draw a conclusion from the result. The findings also suggest that the input data should be more thoroughly considered, as the prediction accuracy is enhanced by the choice of variables and the external information used for training. / Aktiemarknaden är en icke-linjär marknad, men många av de mest kända portföljoptimerings algoritmerna är baserad på linjära modeller. Under de senaste åren har den snabba utvecklingen inom maskininlärning skapat flexibla modeller som kan extrahera information ur komplexa mönster. I det här examensarbetet föreslår vi två sätt att optimera en portfölj, ett där ett neuralt nätverk utvecklas med avseende på multivariata tidsserier och ett annat där vi använder den linjära Markowitz modellen, där vi även lägger ett exponentiellt rörligt medelvärde på prisdatan. Ingångsdatan till vårt neurala nätverk är de dagliga slutpriserna, volymerna och marknadsindikatorer som t.ex. volatilitetsindexet VIX. Utgångsvariablerna kommer vara de predikterade priserna för nästa dag, som sedan bearbetas ytterligare för att producera mätvärden såsom förväntad avkastning, volatilitet och Sharpe ratio. LSTM-modellen producerar en portfölj med avkastning och risk som ligger närmre de verkliga marknadsförhållandena, men däremot gav resultatet ett högt felvärde och det visar att vår LSTM-modell är otillräckligt för att använda som ensamt predikteringssverktyg. Med det sagt så gav det ändå en bättre prediktion när det gäller trender än vad vi antog den skulle göra. Vår slutsats är därför att man bör använda flera neurala nätverk som indikatorer, där var och en är ansvarig för någon specifikt aspekt man vill analysera, och baserat på dessa dra en slutsats. Vårt resultat tyder också på att inmatningsdatan bör övervägas mera noggrant, eftersom predikteringsnoggrannheten.

This thesis is concerned with the problem of predicting the remaining part of the trajectory of a golf ball as it travels through the air where only the three-dimensional position of the ball is captured. The approach taken to solve this problem relied on recurrent neural networks in the form of the long short-term memory networks (LSTM). The motivation behind this choice was that this type of networks had led to state-of-the-art performance for similar problems such as predicting the trajectory of pedestrians. The results show that using LSTMs led to an average reduction of 36.6 % of the error in the predicted impact position of the ball, compared to previous methods based on numerical simulations of a physical model, when the model was evaluated on the same driving range that it was trained on. Evaluating the model on a different driving range than it was trained on leads to improvements in general, but not for all driving ranges, in particular when the ball was captured at a different frequency compared to the data that the model was trained on. This problem was solved to some extent by retraining the model with small amounts of data on the new driving range. / Detta examensarbete har studerat problemet att förutspå den fullständiga bollbanan för en golfboll när den flyger i luften där endast den tredimensionella positionen av bollen observerades. Den typ av metod som användes för att lösa problemet använde sig av recurrent neural networks, i form av long short-term memory nätverk (LSTM). Motivationen bakom detta var att denna typ av nätverk hade lett till goda resultatet för liknande problem. Resultatet visar att använda sig av LSTM nätverk leder i genomsnitt till en 36.6 % förminskning av felet i den förutspådda nedslagsplatsen för bollen jämfört mot tidigare metoder som använder sig av numeriska simuleringar av en fysikalisk modell, om modellen användes på samma golfbana som den tränades på. Att använda en modell som var tränad på en annan golfbana leder till förbättringar i allmänhet, men inte om modellen användes på en golfbana där bollen fångades in med en annan frekvens. Detta problem löstes till en viss mån genom att träna om modellen med lite data från den nya golfbanan.

Machine learning plays a vital role in preventing financial losses within the banking industry, and still, a lot of state of the art and industry-standard approaches within the field neglect rejected customer information and the potential information that they hold to detect similar risk behavior.This thesis explores the possibility of including this information during training and utilizing transactional history through an LSTM to improve the detection of defaults.  The model is structured so an encoder is first trained with or without rejected customers. Virtual distances are then calculated in the embedding space between the accepted customers. These distances are used to create a graph where each node contains an LSTM network, and a GCN passes messages between connected nodes. The model is validated using two datasets, one public Taiwan dataset and one private Swedish one provided through the collaborative company. The Taiwan dataset used 8000 data points with a 50/50 split in labels. The Swedish dataset used 4644 with the same split.  Multiple metrics were used to validate the impact of the rejected customers and the impact of using time-series data instead of static features. For the encoder part, reconstruction error was used to measure the difference in performance. When creating the edges, the homogeny of the neighborhoods and if a node had a majority of the same labeled neighbors as itself were determining factors, and for the classifier, accuracy, f1-score, and confusion matrix were used to compare results. The results of the work show that the impact of rejected customers is minor when it comes to changes in predictive power. Regarding the effects of using time-series information instead of static features, we saw a comparative result to XGBoost on the Taiwan dataset and an improvement in the predictive power on the Swedish dataset. The results also show the importance of a well-defined virtual distance is critical to the classifier's performance.

Sepsis is a life-threatening condition that neonatal patients are especially susceptible to. Fortunately, improved bedside monitoring has enabled the collection and use of continuous vital signs data for the purpose of detecting conditions such as sepsis. While current research has found some success in reducing mortality in neonatal intensive care units with linear directly interpretable models, such as logistic regression, accurate detection of sepsis from inherently noisy time-series data still remains a challenge. Furthermore, previous research has generally relied on pre-defined features extracted from rawvital signs data, which may not be optimal for the detection task. Therefore, assessing the overall feasibility of sequential deep learning models, such as recurrent and convolutional models, could improve the results of current research. This task was tackled in three phases. Firstly, baseline scores were established with a logistic regression model. Secondly, three common recurrent classifiers were tested on pre-defined window based features and compared with each other. Thirdly, a convolutional architecture with a recurrent and non-recurrent classifier was tested on raw low frequency (1Hz) signals in order to examine their capability to automatically extract features from the data. The final results from all phases were compared with each other. Results show that recurrent classifiers trained on pre-defined features do outperform automatic feature extraction with the convolutional models. The best model was based on a long-short term memory unit that achieved an area under the characteristic receiver operating unit curve of 0.806, and outperformed the established baseline results. In comparison with previous research, said model performed on par with the examined simple interpretable baseline models. The low results can likely be attributed to a insufficient sample size of patients with sepsis for the examined models and sub-optimal hyperparameter optimization due to the number of possible configurations. Further avenues of research include examination of high frequency data and more complex models for automatic feature extraction. / Sepsis är ett livshotande tillstånd som neonatala patienter är särskilt mottagliga för. Lyckligtvis har förbättrad patientmonitorering möjliggjort kontinuerlig insamling och andvänding av vitalparametrar i syfte att upptäcka tillstånd som sepsis. Medan aktuell forskning har funnit viss framgång i att minska dödligheten på neonatala intensivvårdsavdelningar med hjälp av linjära tolkbara modeller, såsom logistisk regression, är noggrann detektering av sepsis från brusig tidsseriedata fortfarande en utmaning. Dessutom har tidigare forskning i allmänhet förlitat sig på fördefinierade prediktorer extraherade från rå vitalparameterdata, som kanske inte är optimala för detektionsuppgiften. På grund av detta kan en bedömning av den övergripande användbarheten av sekventiella modeller för djupinlärning, såsom RNN- och CNN-modeller, förbättra resultaten av aktuell forskning. Denna uppgift tacklades i tre faser. Först och främst etablerades baslinjeresultat med en logistisk regressionsmodell. För det andra testades tre RNN-baserad klassificerare på data med fördefinierade fönsterbaserade prediktorer och jämfördes med varandra. För det tredje testades en CNN-arkitektur med både en RNN-klassificerare och MLP-klassificerare på råa lågfrekventa (1Hz) signaler för att undersöka deras förmåga att automatiskt extrahera egna prediktorer från datan. Slutresultaten från alla faser jämfördes med varandra. Resultaten visar att RNN-klassificerare som tränats på fördefinierade prediktorer överträffar automatisk extraktion av prediktorer med CNN-modellerna. Den bäst presterande modellen baserades på en långtidsminnesenhet som uppnådde en AUROC på 0.806, och överträffade de etablerade baslinjeresultaten. I jämförelse med tidigare forskning uppnådde ifrågavarande modell lika hög prestation som de väl undersökta enklare tolkbara baslinjemodellerna. De låga resultaten kan sannolikt tillskrivas en otillräcklig provstorlek av patienter med sepsis för de undersökta modellerna och suboptimal hyperparameteroptimering på grund av antalet möjliga konfigurationer. Ytterligare forskningsvägar inkluderar undersökning av högfrekventa data och mer komplexa modeller för automatisk extraktion av prediktorer.

With the growing installation of home photovoltaics, traditional energy trading is evolving from a unidirectional utility-to-consumer model into a more distributed peer-to-peer paradigm. Besides, with the development of building energy management platforms and demand response-enabled smart devices, energy consumption saved, known as negawatt-hours, has also emerged as another commodity that can be exchanged. Users may tune their heating, ventilation, and air conditioning (HVAC) system setpoints to adjust building hourly energy consumption to generate negawatt-hours. Both photovoltaic (PV) energy and negawatt-hours are two major resources of peer-to-peer electricity trading. Blockchain has been touted as an enabler for trustworthy and reliable peer-to-peer trading to facilitate the deployment of such distributed electricity trading through encrypted processes and records. Unfortunately, blockchain cannot fully detect anomalous participant behaviors or malicious inputs to the network. Consequentially, end-user anomaly detection is imperative in enhancing trust in peer-to-peer electricity trading. This dissertation introduces machine learning-based anomaly detection techniques in peer-to-peer PV energy and negawatt-hour trading. This can help predict the next hour's PV energy and negawatt-hours available and flag potential anomalies when submitted bids. As the traditional energy trading market is agnostic to tangible real-world resources, developing, evaluating, and integrating machine learning forecasting-based anomaly detection methods can give users knowledge of reasonable bid offer quantity. Suppose a user intentionally or unintentionally submits extremely high/low bids that do not match their solar panel capability or are not backed by substantial negawatt-hours and PV energy resources. Some anomalies occur because the participant's sensor is suffering from integrity errors. At the same time, some other abnormal offers are maliciously submitted intentionally to benefit attackers themselves from market disruption. In both cases, anomalies should be detected by the algorithm and rejected by the market. Artificial Neural Networks (ANN), Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), and Convolutional Neural Network (CNN) are compared and studied in PV energy and negawatt-hour forecasting. The semi-supervised anomaly detection framework is explained, and its performance is demonstrated. The threshold values of anomaly detection are determined based on the model trained on historical data. Besides ambient weather information, HVAC setpoint and building occupancy are input parameters to predict building hourly energy consumption in negawatt-hour trading. The building model is trained and managed by negawatt-hour aggregators. CO2 monitoring devices are integrated into the cloud-based smart building platform BEMOSS™ to demonstrate occupancy levels, further improving building load forecasting accuracy in negawatt-hour trading. The relationship between building occupancy and CO2 measurement is analyzed. Finally, experiments based on the Hyperledger platform demonstrate blockchain-based peer-to-peer energy trading and how the platform detects anomalies. / Doctor of Philosophy / The modern power grid is transforming from unidirectional to transactive power systems. Distributed peer-to-peer (P2P) energy trading is becoming more and more popular. Rooftop PV energy and negawatt-hours as two main sources of electricity assets are playing important roles in peer-to-peer energy trading. It enables the building owner to join the electricity market as both energy consumer and producer, also named prosumer. While P2P energy trading participants are usually un-informed and do not know how much energy they can generate during the next hour. Thus, a system is needed to guide the participant to submit a reasonable amount of PV energy or negawatt-hours to be supplied. This dissertation develops a machine learning-based anomaly detection model for an energy trading platform to detect the reasonable PV energy and negawatt-hours available for the next hour's electricity trading market. The anomaly detection performance of this framework is analyzed. The building load forecasting model used in negawatt-hour trading also considers the effect of building occupancy level and HVAC setpoint adjustment. Moreover, the implication of CO2 measurement devices to monitor building occupancy levels is demonstrated. Finally, a simple Hyperledger-based electricity trading platform that enables participants to sell photovoltaic solar energy/ negawatt-hours to other participants is simulated to demonstrate the potential benefits of blockchain.

Predicting the behavior of financial markets is largely an unsolved problem. The problem hasbeen approached with many different methods ranging from binary logic, statisticalcalculations and genetic algorithms. In this thesis, the problem is approached with a machinelearning method, namely the Long Short-Term Memory (LSTM) variant of Recurrent NeuralNetworks (RNNs). Recurrent neural networks are artificial neural networks (ANNs)—amachine learning algorithm mimicking the neural processing of the mammalian nervoussystem—specifically designed for time series sequences. The thesis investigates the capabilityof the LSTM in modeling financial market behavior as well as compare it to the traditionalRNN, evaluating their performances using various measures. / Prediktion av den finansiella marknadens beteende är i stort ett olöst problem. Problemet hartagits an på flera sätt med olika metoder så som binär logik, statistiska uträkningar ochgenetiska algoritmer. I den här uppsatsen kommer problemet undersökas medmaskininlärning, mer specifikt Long Short-Term Memory (LSTM), en variant av rekurrentaneurala nätverk (RNN). Rekurrenta neurala nätverk är en typ av artificiellt neuralt nätverk(ANN), en maskininlärningsalgoritm som ska efterlikna de neurala processerna hos däggdjursnervsystem, specifikt utformat för tidsserier. I uppsatsen undersöks kapaciteten hos ett LSTMatt modellera finansmarknadens beteenden och jämförs den mot ett traditionellt RNN, merspecifikt mäts deras effektivitet på olika vis.

Modelling music with mathematical and statistical methods in general, and with neural networks in particular, has a long history and has been well explored in the last decades. Exactly when the first attempt at strictly systematic music took place is hard to say; some would say in the days of Mozart, others would say even earlier, but it is safe to say that the field of algorithmic composition has a long history. Even though composers have always had structure and rules as part of the writing process, implicitly or explicitly, following rules at a stricter level was well investigated in the middle of the 20th century at which point also the first music writing computer program based on mathematics was implemented. This work in computer science focuses on the history of musical composition with computers, also known as algorithmic composition, using machine learning and neural networks and consists of two parts: a literature survey covering in-depth the last decades in the field from which is drawn inspiration and experience to construct MahlerNet, a neural network based on the previous architectures MusicVAE, BALSTM, PerformanceRNN and BachProp, capable of modelling polyphonic symbolic music with up to 23 instruments. MahlerNet is a new architecture that uses a custom preprocessor with musical heuristics to normalize and filter the input and output files in MIDI format into a data representation that it uses for processing. MahlerNet, and its preprocessor, was written altogether for this project and produces music that clearly shows musical characteristics reminiscent of the data it was trained on, with some long-term structure, albeit not in the form of motives and themes. / Matematik och statistik i allmänhet, och maskininlärning och neurala nätverk i synnerhet, har sedan långt tillbaka använts för att modellera musik med en utveckling som kulminerat under de senaste decennierna. Exakt vid vilken historisk tidpunkt som musikalisk komposition för första gången tillämpades med strikt systematiska regler är svårt att säga; vissa skulle hävda att det skedde under Mozarts dagar, andra att det skedde redan långt tidigare. Oavsett vilket, innebär det att systematisk komposition är en företeelse med lång historia. Även om kompositörer i alla tider följt strukturer och regler, medvetet eller ej, som en del av kompositionsprocessen började man under 1900-talets mitt att göra detta i högre utsträckning och det var också då som de första programmen för musikalisk komposition, baserade på matematik, kom till. Den här uppsatsen i datateknik behandlar hur musik historiskt har komponerats med hjälp av datorer, ett område som också är känt som algoritmisk komposition. Uppsatsens fokus ligger på användning av maskininlärning och neurala nätverk och består av två delar: en litteraturstudie som i hög detalj behandlar utvecklingen under de senaste decennierna från vilken tas inspiration och erfarenheter för att konstruera MahlerNet, ett neuralt nätverk baserat på de tidigare modellerna MusicVAE, BALSTM, PerformanceRNN och BachProp. MahlerNet kan modellera polyfon musik med upp till 23 instrument och är en ny arkitektur som kommer tillsammans med en egen preprocessor som använder heuristiker från musikteori för att normalisera och filtrera data i MIDI-format till en intern representation. MahlerNet, och dess preprocessor, är helt och hållet implementerade för detta arbete och kan komponera musik som tydligt uppvisar egenskaper från den musik som nätverket tränats på. En viss kontinuitet finns i den skapade musiken även om det inte är i form av konkreta teman och motiv.

As an efficient energy storage technology, lithium-ion batteries play a key role in the ongoing electrification of the mobility sector. However, the required modelbased design process, including hardware in the loop solutions, demands precise battery models. In this work, an encoder-decoder model framework based on recurrent neural networks is developed and trained directly on unstructured battery data to replace time consuming characterisation tests and thus simplify the modelling process. A manifold pseudo-random bit stream dataset is used for model training and validation. A mean percentage error (MAPE) of 0.30% for the test dataset attests the proposed encoder-decoder model excellent generalisation capabilities. Instead of the recursive one-step prediction prevalent in the literature, the stage-wise trained encoder-decoder framework can instantaneously predict the battery voltage response for 2000 time steps and proves to be 120 times more time-efficient on the test dataset. Accuracy, generalisation capability and time efficiency of the developed battery model enable a potential online anomaly detection, power or range prediction. The fact that, apart from the initial voltage level, the battery model only relies on the current load as input and thus requires no estimated variables such as the state-of-charge (SOC) to predict the voltage response holds the potential of a battery ageing independent LIB modelling based on raw BMS signals. The intrinsically ageingindependent battery model is thus suitable to be used as a digital battery twin in virtual experiments to estimate the unknown battery SOH on purely BMS data basis.

Title: Neural networks for automatic speaker, language, and sex identifica- tion Author: Bich-Ngoc Do Department: Institute of Formal and Applied Linguistics Supervisor: Ing. Mgr. Filip Jurek, Ph.D., Institute of Formal and Applied Linguistics and Dr. Marco Wiering, Faculty of Mathematics and Natural Sciences, University of Groningen Abstract: Speaker recognition is a challenging task and has applications in many areas, such as access control or forensic science. On the other hand, in recent years, deep learning paradigm and its branch, deep neural networks have emerged as powerful machine learning techniques and achieved state-of- the-art in many fields of natural language processing and speech technology. Therefore, the aim of this work is to explore the capability of a deep neural network model, recurrent neural networks, in speaker recognition. Our pro- posed systems are evaluated on TIMIT corpus using speaker identification task. In comparison with other systems in the same test conditions, our systems could not surpass reference ones due to the sparsity of validation data. In general, our experiments show that the best system configuration is a combination of MFCCs with their dynamic features and a recurrent neural network model. We also experiment recurrent neural networks and convo- lutional neural...