Credit scoring is critical for banks to evaluate new loan applications and monitor existing customers. Machine learning has been extensively researched for this case; however, the adoption of machine learning methods is minimal in financial risk management. The primary reason is that algorithms are viewed as ‘black box models’ and cannot satisfy regulatory requirements. While deep learning methods such as LSTM have been evaluated for behavioural credit scoring based on performance, research has not holistically evaluated these models on performance and explainability. To answer the research question, How can traditional machine learning and deep learning methods conform with regulatory guidelines for explainable artificial intelligence (XAI), and are they preferable to benchmark methods? this thesis used a public customer credit card dataset to compare the performance and explainability of machine learning and deep learning models against the benchmark statistical model linear regression. Model performance was evaluated using ROC-AUC, accuracy, Brier scores, F1 scores and the G-mean. The McNemar test evaluated whether, through pairwise comparison, the model performances were statistically different. The models were then evaluated on whether local and global explanations could be ascertained using feature/permutation importance and SHAP. The results found that neither the machine learning model, XGBoost, nor the deep learning model, LSTM, produced a statistically superior performance from the benchmark model. While there were performance improvements, only the machine learning model using post-hoc methods could produce local and global explanations. Given the strict regulatory environment, it is understandable that banks are hesitant to implement machine learning or deep learning models that lack the adequate levels of explainability regulators require.

In this thesis, anomalies are defined as data points whose value differs significantly from the normal pattern of the data set. Anomalousobservations on time series measured on satellites has a growing need of being detected directly on board the space-orbit systems to for example prevent malfunction and have efficient data management. Unibap's service Spacecloud Framework (SCFW) is developed to allow the deployment of machine learning applications directly on the satellite systems. Neural Networks (NNs) is therefore a candidate for the possibility to predict anomalies on satellite time series. The work described in this reportaims to implement and create a benchmark for Convolutional Autoencoder NN (CNN) and a Long Short-term Memory Autoencoder NN (LSTM). These implementations are used to determine which NN can be applied in Unibap's SCFW and detect anomalies with accuracy.  The NNs are trained and tested using a public data-sets which containreal and artificial time-series with labelled anomalies. The anomaliesare detected by reconstructing the time series and creating a threshold between the output and the input. The algorithms classify a data pointas an anomaly if it lies above the threshold. The networks are evaluated based on accuracy, execution time and size, to assess whether they are suited for implementation in SCFW. The results from the NNs indicatethat CNN is best suited for further application. On this basis, anattempt to implement CNN in SCFW is performed, but failed due to time and documentation limitations. Therefore, further research is needed to identify whether CNN can be implemented in SCFW and successfully detect anomalies.

During the past years an increased focus on the development of autonomous solutions has resulted in driverless vehicles being used in numerous industries. Volvo Construction Equipment is currently developing the TA15, an autonomous hauler part of a larger transport solution. The transition to autonomous haulers have further increased the need for improved system condition monitoring in the strive for increased operational time. A method aiming to identify and isolate faults in the hydraulic steering system on the TA15 was therefore investigated in this thesis. Using fault tree analysis, five faults considered to be of importance regarding steering performance were selected. Two different methods for detecting the faults were compared to each other, data-driven and model based. Out of the two, data-driven was selected as the method of choice due to high modularity and relative simplicity regarding implementation. The data-driven approach consisted of Feed-Forward and Long Short Term Memory networks where the suitable inputs were decided to be a combination of pressure and position signals. Utilizing a simulation model of the steering system validated against the TA15, the selected faults were induced in the simulated system with various severity. Training the networks to classify and estimate fault severity in the simulated model resulted in satisfactory results using both networks. It was however concluded that in contrary to the Feed-Forward network, the LSTM network could achieve good performance using less amount of sensors. Although the diagnostic method showed promising result on a simulation model, test on the real TA15 needs to be performed in order to properly evaluate the method. The advantage of using a data-driven approach was specially noticeable when comparisons were made to the model based approach. The data-driven approach relies on labeling data rather than complete system knowledge. Meaning that the method developed therefore could be applied on practically any hydraulic system in construction equipment by changing the training data.

Membrane bioreactors (MBRs) have proven to be an extremely effective wastewater treatment process combining ultrafiltration with biological processes to produce high-quality effluent. However, one of the major drawbacks to this technology is membrane fouling – an inevitable process that reduces permeate production and increases operating costs. The prediction of membrane fouling in MBRs is important because it can provide decision support to wastewater treatment plant (WWTP) operators. Currently, mechanistic models are often used to estimate transmembrane pressure (TMP), which is an indicator of membrane fouling, but their performance is not always satisfactory. In this research, existing mechanistic and data-driven models used for membrane fouling are investigated. Data-driven machine learning techniques consisting of random forest (RF), artificial neural network (ANN), and long-short term memory network (LSTM) are used to build models to predict transmembrane pressure (TMP) at various stages of the MBR production cycle. The models are built with 4 years of high-resolution data from a confidential full-scale municipal WWTP. The model performances are examined using statistical measures such as coefficient of determination (R2), root mean squared error, mean absolute percentage error, and mean squared error. The results show that all models provide reliable predictions while the RF models have the best predictive accuracy when compared to the ANN and LSTM models. The corresponding R2 values for RF when predicting before, during, and after back pulse TMP are 0.996, 0.927, and 0.996, respectively. Model uncertainty (including hyperparameter and algorithm uncertainty) is quantified to determine the impact of hyperparameter tuning and the variance of extreme predictions caused by algorithm choice. The ANN models are most impacted by hyperparameter tuning and have the highest variability when predicting extreme values within each model’s respective hyperparameter range. The proposed models can be useful tools in providing decision support to WWTP operators employing fouling mitigation strategies, which can potentially lead to better operation of WWTPs and reduced costs. / Thesis / Master of Applied Science (MASc)

This paper aims to reverse engineer the tradingstrategy of an actively managed mutual fund by identifyingtechnical patterns in their trading. Investment strategies formany institutional investors consists of both fundamental andtechnical analysis. The purpose of the paper is to explore towhich extent the latter can be used to predict the trading actionsby taking some commonly used technical indicators as input invarious machine learning algorithms to assess patterns betweenthem and the trading of the fund. Furthermore, the technicalindicators’ ability to predict future prices is analysed using thesame methods. The results are not sufficiently clear to suggestthat the fund uses technical indicators to begin with, let alonewhich ones. As for the prediction of future prices, the technicalindicators appear to have some predictive ability. / Syftet med denna rapport är att prediktera handeln i en aktivt förvaltad aktiefond med hjälp av fyra maskininlärningsmetoder. Investeringsstrategier kombinerar i regel två analysmetoder, fundamental respektive teknisk analys. Avsikten med rapporten är att utforska huruvida det sistnämnda kan användas för att förutspå fondens handel genom att använda ett antal vanligt förekommande tekniska indikatorer och medelst maskininlärningsmetoder söka efter mönster mellan dessa och handeln. Vidare innefattar även studien en analys över hur väl tekniska indikatorer predikterar upprespektive nedgångar på aktiepriser. Vad gäller investeringsstrategierna återfanns inga tydliga samband mellan de utvalda indikatorerna och transaktionerna. Resultaten för andra delen av studien tyder på viss prediktiv förmåga för tekniska indikatorer på marknadsrörelser. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

Water monitoring is an essential process that managesthe well-being of freshwater ecosystems. However, it isgenerally an inefficient process as most data collection is donemanually. By combining wireless sensor technology and machinelearning techniques, projects such as iWater aim to modernizecurrent methods. The purpose of the iWater project is to developa network of smart sensors capable of collecting and analyzingwater quality-related data in real time.To contribute to this goal, a comparative study between theperformance of a centralized machine learning algorithm thatis currently used, and a distributed model based on a federatedlearning algorithm was done. The data used for training andtesting both models was collected by a wireless sensor developedby the iWater project. The centralized algorithm was used asthe basis for the developed distributed model. Due to lack ofsensors, the distributed model was simulated by down-samplingand dividing the sensor data into six data sets representing anindividual sensor. The results are similar for both models andthe developed algorithm reaches an accuracy of 98.41 %. / Vattenövervakning är en nödvändig processför att få inblick i sötvattensekosystems välmående. Dessvärreär det en kostsam och tidskrävande process då insamling avdata vanligen görs manuellt. Genom att kombinera trådlössensorteknologi och maskininlärnings algoritmer strävar projektsom iWater mot att modernisera befintliga metoder.Syftet med iWater är att skapa ett nätverk av smarta sensorersom kan samla in och analysera vattenkvalitetsrelaterade datai realtid. För att bidra till projektmålet görs en jämförandestudie mellan den prediktiva noggrannheten hos en centraliseradmaskininlärningsalgoritm, som i nuläget används, och endistribuerad modell baserad på federerat lärande. Data somanvänds för träning och testning av båda modellerna samladesin genom en trådlös sensor utvecklad inom iWater-projektet.Den centraliserade algoritmen användes som grund för denutvecklade distribuerade modellen. På grund av brist på sensorersimulerades den distribuerade modellen genom nedprovtagningoch uppdelning av data i sex datamängder som representerarenskilda sensorer. Resultaten för båda modellerna var liknandeoch den utvecklade algoritmen har en noggrannhet på 98.41 % / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

With the rapid increase of Internet of Things-devices(IoT), demand for new machine learning algorithms and modelshas risen. The focus of this project is implementing a federatedlearning (FL) algorithm to detect anomalies in measurementsmade by a water monitoring IoT-sensor. The FL algorithm trainsacross a collection of decentralized IoT-devices, each using thelocal data acquired from the specific sensor. The local machinelearning models are then uploaded to a mutual server andaggregated into a global model. The global model is sent back tothe sensors and is used as a template when training starts againlocally. In this project, we only have had access to one physicalsensor. This has forced us to virtually simulate sensors. Thesimulation was done by splitting the data gathered by the onlyexisting sensor. To deal with the long, sequential data gatheredby the sensor, a long short-term memory (LSTM) network wasused. This is a special type of artificial neural network (ANN)capable of learning long-term dependencies. After analyzing theobtained results it became clear that FL has the potential toproduce good results, provided that more physical sensors aredeployed. / I samband med den snabba ökningen avInternet of Things-enheter (IoT) har efterfrågan på nya algoritmeroch modeller för maskininlärning ökat. Detta projektfokuserar på att implementera en federated learning (FL) algoritmför att detektera avvikelser i mätdata från en sensorsom övervakar vattenkvaliteten. FL algoritmen tränar en samlingdecentraliserade IoT-enheter, var och en med hjälp av lokaldata från sensorn i fråga. De lokala maskininlärningsmodellernaladdas upp till en gemensam server och sammanställs till englobal modell. Den globala modellen skickas sedan tillbaka tillsensorerna och används som mall när den lokala träningen börjarigen. I det här projektet hade vi endast tillgång till en fysisksensor. Vi har därför varit tvungna att simulera sensorer. Dettagjordes genom att dela upp datamängden som samlats in frånden fysiska sensorn. För att hantera den långa sekventiella dataanvänds ett long short-term memory (LSTM) nätverk. Detta ären speciell typ av artificiellt neuronnät (ANN) som är kapabeltatt minnas mönster under en längre tid. Efter att ha analyseratresultaten blev det tydligt att FL har potentialen att produceragoda resultat, givet att fler fysiska sensorer implementeras. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

Anomaly detection in Internet of Things (IoT) systems has become an increasingly popular field of research as the number of IoT devices proliferate year over year. Recent research often relies on machine learning algorithms to classify sensor readings directly. However, this approach leads to solutions being non-portable and unable to be applied to varying IoT platform infrastructure, as they are trained with sensor data specific to one configuration. Moreover, sensors generate varying amounts of non-standard data which complicates model training and limits generalization. This research focuses on addressing these problems in three ways a) the creation of an IoT Testbed which is configurable and parameterizable for dataset generation, b) the usage of system performance metrics as the dataset for training the anomaly classifier which ensures a fixed dataset size, and c) the application of Symbolic Aggregate Approximation (SAX) to encode patterns in system performance metrics which allows our trained Long Short-Term Memory (LSTM) model to classify anomalies agnostic to the underlying system configuration. Our devised IoT Testbed provides a lightweight setup for data generation which directly reflects some of the most pertinent components of Industry 4.0 pipelines including a MQTT Broker, Apache Kafka, and Apache Cassandra. Additionally, our proposed solution provides improved portability over state-of-the-art models while standardizing the required training data. Results demonstrate the effectiveness of utilizing symbolized performance metrics as we were able to achieve accuracies of 95.87%, 87.33%, and 87.47% for three different IoT system configurations. The latter two accuracies represent the model’s ability to be generalized to datasets generated from differing system configurations.

This study proposes an accuracy comparison of two of the best performing machine learning algorithms in natural language processing, the Bayesian Network and the Long Short-Term Memory (LSTM) Recurrent Neural Network, in detecting Alzheimer’s disease symptoms in conversation transcripts. Because of the current global rise of life expectancy, the number of seniors affected by Alzheimer’s disease worldwide is increasing each year. Early detection is important to ensure that affected seniors take measures to relieve symptoms when possible or prepare plans before further cognitive decline occurs. Literature shows that natural language processing can be a valid tool for early diagnosis of the disease. This study found that mild dementia and possible Alzheimer’s can be detected in conversation transcripts with promising results, and that the LSTM is particularly accurate in said detection, reaching an accuracy of 86.5% on the chosen dataset. The Bayesian Network classified with an accuracy of 72.1%. The study confirms the effectiveness of a natural language processing approach to detecting Alzheimer’s disease.

In this paper a novel speaker recognition system is introduced. Automated speaker recognition has become increasingly popular to aid in crime investigations and authorization processes with the advances in computer science. Here, a recurrent neural network approach is used to learn to identify ten speakers within a set of 21 audio books. Audio signals are processed via spectral analysis into Mel Frequency Cepstral Coefficients that serve as speaker specific features, which are input to the neural network. The Long Short-Term Memory algorithm is examined for the first time within this area, with interesting results. Experiments are made as to find the optimum network model for the problem. These show that the network learns to identify the speakers well, text-independently, when the recording situation is the same. However the system has problems to recognize speakers from different recordings, which is probably due to noise sensitivity of the speech processing algorithm in use.