<div>Textual emotion classification is a task in affective AI that branches from sentiment analysis and focuses on identifying emotions expressed in a given text excerpt. </div><div>It has a wide variety of applications that improve human-computer interactions, particularly to empower computers to understand subjective human language better. </div><div>Significant research has been done on this task, but very little of that research leverages one of the most emotion-bearing symbols we have used in modern communication: Emojis.</div><div>In this thesis, we propose several transformer-based models for emotion classification that processes emojis as input tokens and leverages pretrained models and uses them</div><div>, a model that processes Emojis as textual inputs and leverages DeepMoji to generate affective feature vectors used as reference when aggregating different modalities of text encoding. </div><div>To evaluate ReferEmo, we experimented on the SemEval 2018 and GoEmotions datasets, two benchmark datasets for emotion classification, and achieved competitive performance compared to state-of-the-art models tested on these datasets. Notably, our model performs better on the underrepresented classes of each dataset.</div>

Predicting stock market trends is a complex task due to the inherent volatility and unpredictability of financial markets. Nevertheless, accurate forecasts are of critical importance to investors, financial analysts, and stakeholders, as they directly inform decision-making processes and risk management strategies associated with financial investments. Inaccurate forecasts can lead to notable financial consequences, emphasizing the crucial and demanding task of developing models that provide accurate and trustworthy predictions. This article addresses this challenging problem by utilizing a long-short term memory (LSTM) model to predict stock market developments. The study undertakes a thorough analysis of the LSTM model's performance across multiple datasets, critically examining the impact of different timespans and epochs on the accuracy of its predictions. Additionally, a comparison is made with a support vector regression (SVR) model using the same datasets and timespans, which allows for a comprehensive evaluation of the relative strengths of the two techniques. The findings offer insights into the capabilities and limitations of both models, thus paving the way for future research in stock market prediction methodologies. Crucially, the study reveals that larger datasets and an increased number of epochs can significantly enhance the LSTM model's performance. Conversely, the SVR model exhibits significant challenges with overfitting. Overall, this research contributes to ongoing efforts to improve financial prediction models and provides potential solutions for individuals and organizations seeking to make accurate and reliable forecasts of stock market trends.

Financial market forecasting is a challenging and complex task due to the sensitivity of the market to various factors such as political, economic, and social factors. However, recent advances in machine learning and computation technology have led to an increased interest in using deep learning for forecasting financial data. One the one hand, the famous efficient market hypothesis states that the market is so efficient that no one can consistently benefit from it, and the random walk theory suggests that asset prices are unpredictable based on historical data. On the other hand, previous research has shown that financial time series can be forecasted to some extent using artificial neural networks (ANNs). Despite being a relatively new addition to financial research with less study than the traditional models such as moving averages and linear regression models, ANNs have been shown to outperform the traditional models to some extent. Hence, considering the efficient market hypothesis and the random walk theory, there is a knowledge gap on whether neural networks can be used for financial time series prediction. This paper explores the use of ANNs, specifically recurrent neural networks, to predict financial time series data using a long short-term memory (LSTM) network model. The study will employ an experimental research strategy to construct and test an LSTM model to predict financial time series data, with the aim of examining its performance and evaluating it relative to other models and methods. For evaluating its performance, evaluation metrics are computed and the model is compared with a constructed simple moving average (SMA) model as well as other models in existing studies. The paper also explores the application and processing of transformed financial data, where it was found that achieving stationarity by data transformation was not necessary for the LSTM model to perform better. The study also found that the LSTM model outperformed the SMA model when hyperparameters were set to capture long-term dependencies. However, in the short-term, the SMA model outperformed the LSTM model.

Credit scoring is critical for banks to evaluate new loan applications and monitor existing customers. Machine learning has been extensively researched for this case; however, the adoption of machine learning methods is minimal in financial risk management. The primary reason is that algorithms are viewed as ‘black box models’ and cannot satisfy regulatory requirements. While deep learning methods such as LSTM have been evaluated for behavioural credit scoring based on performance, research has not holistically evaluated these models on performance and explainability. To answer the research question, How can traditional machine learning and deep learning methods conform with regulatory guidelines for explainable artificial intelligence (XAI), and are they preferable to benchmark methods? this thesis used a public customer credit card dataset to compare the performance and explainability of machine learning and deep learning models against the benchmark statistical model linear regression. Model performance was evaluated using ROC-AUC, accuracy, Brier scores, F1 scores and the G-mean. The McNemar test evaluated whether, through pairwise comparison, the model performances were statistically different. The models were then evaluated on whether local and global explanations could be ascertained using feature/permutation importance and SHAP. The results found that neither the machine learning model, XGBoost, nor the deep learning model, LSTM, produced a statistically superior performance from the benchmark model. While there were performance improvements, only the machine learning model using post-hoc methods could produce local and global explanations. Given the strict regulatory environment, it is understandable that banks are hesitant to implement machine learning or deep learning models that lack the adequate levels of explainability regulators require.

In this thesis, anomalies are defined as data points whose value differs significantly from the normal pattern of the data set. Anomalousobservations on time series measured on satellites has a growing need of being detected directly on board the space-orbit systems to for example prevent malfunction and have efficient data management. Unibap's service Spacecloud Framework (SCFW) is developed to allow the deployment of machine learning applications directly on the satellite systems. Neural Networks (NNs) is therefore a candidate for the possibility to predict anomalies on satellite time series. The work described in this reportaims to implement and create a benchmark for Convolutional Autoencoder NN (CNN) and a Long Short-term Memory Autoencoder NN (LSTM). These implementations are used to determine which NN can be applied in Unibap's SCFW and detect anomalies with accuracy.  The NNs are trained and tested using a public data-sets which containreal and artificial time-series with labelled anomalies. The anomaliesare detected by reconstructing the time series and creating a threshold between the output and the input. The algorithms classify a data pointas an anomaly if it lies above the threshold. The networks are evaluated based on accuracy, execution time and size, to assess whether they are suited for implementation in SCFW. The results from the NNs indicatethat CNN is best suited for further application. On this basis, anattempt to implement CNN in SCFW is performed, but failed due to time and documentation limitations. Therefore, further research is needed to identify whether CNN can be implemented in SCFW and successfully detect anomalies.

During the past years an increased focus on the development of autonomous solutions has resulted in driverless vehicles being used in numerous industries. Volvo Construction Equipment is currently developing the TA15, an autonomous hauler part of a larger transport solution. The transition to autonomous haulers have further increased the need for improved system condition monitoring in the strive for increased operational time. A method aiming to identify and isolate faults in the hydraulic steering system on the TA15 was therefore investigated in this thesis. Using fault tree analysis, five faults considered to be of importance regarding steering performance were selected. Two different methods for detecting the faults were compared to each other, data-driven and model based. Out of the two, data-driven was selected as the method of choice due to high modularity and relative simplicity regarding implementation. The data-driven approach consisted of Feed-Forward and Long Short Term Memory networks where the suitable inputs were decided to be a combination of pressure and position signals. Utilizing a simulation model of the steering system validated against the TA15, the selected faults were induced in the simulated system with various severity. Training the networks to classify and estimate fault severity in the simulated model resulted in satisfactory results using both networks. It was however concluded that in contrary to the Feed-Forward network, the LSTM network could achieve good performance using less amount of sensors. Although the diagnostic method showed promising result on a simulation model, test on the real TA15 needs to be performed in order to properly evaluate the method. The advantage of using a data-driven approach was specially noticeable when comparisons were made to the model based approach. The data-driven approach relies on labeling data rather than complete system knowledge. Meaning that the method developed therefore could be applied on practically any hydraulic system in construction equipment by changing the training data.

Membrane bioreactors (MBRs) have proven to be an extremely effective wastewater treatment process combining ultrafiltration with biological processes to produce high-quality effluent. However, one of the major drawbacks to this technology is membrane fouling – an inevitable process that reduces permeate production and increases operating costs. The prediction of membrane fouling in MBRs is important because it can provide decision support to wastewater treatment plant (WWTP) operators. Currently, mechanistic models are often used to estimate transmembrane pressure (TMP), which is an indicator of membrane fouling, but their performance is not always satisfactory. In this research, existing mechanistic and data-driven models used for membrane fouling are investigated. Data-driven machine learning techniques consisting of random forest (RF), artificial neural network (ANN), and long-short term memory network (LSTM) are used to build models to predict transmembrane pressure (TMP) at various stages of the MBR production cycle. The models are built with 4 years of high-resolution data from a confidential full-scale municipal WWTP. The model performances are examined using statistical measures such as coefficient of determination (R2), root mean squared error, mean absolute percentage error, and mean squared error. The results show that all models provide reliable predictions while the RF models have the best predictive accuracy when compared to the ANN and LSTM models. The corresponding R2 values for RF when predicting before, during, and after back pulse TMP are 0.996, 0.927, and 0.996, respectively. Model uncertainty (including hyperparameter and algorithm uncertainty) is quantified to determine the impact of hyperparameter tuning and the variance of extreme predictions caused by algorithm choice. The ANN models are most impacted by hyperparameter tuning and have the highest variability when predicting extreme values within each model’s respective hyperparameter range. The proposed models can be useful tools in providing decision support to WWTP operators employing fouling mitigation strategies, which can potentially lead to better operation of WWTPs and reduced costs. / Thesis / Master of Applied Science (MASc)

This paper aims to reverse engineer the tradingstrategy of an actively managed mutual fund by identifyingtechnical patterns in their trading. Investment strategies formany institutional investors consists of both fundamental andtechnical analysis. The purpose of the paper is to explore towhich extent the latter can be used to predict the trading actionsby taking some commonly used technical indicators as input invarious machine learning algorithms to assess patterns betweenthem and the trading of the fund. Furthermore, the technicalindicators’ ability to predict future prices is analysed using thesame methods. The results are not sufficiently clear to suggestthat the fund uses technical indicators to begin with, let alonewhich ones. As for the prediction of future prices, the technicalindicators appear to have some predictive ability. / Syftet med denna rapport är att prediktera handeln i en aktivt förvaltad aktiefond med hjälp av fyra maskininlärningsmetoder. Investeringsstrategier kombinerar i regel två analysmetoder, fundamental respektive teknisk analys. Avsikten med rapporten är att utforska huruvida det sistnämnda kan användas för att förutspå fondens handel genom att använda ett antal vanligt förekommande tekniska indikatorer och medelst maskininlärningsmetoder söka efter mönster mellan dessa och handeln. Vidare innefattar även studien en analys över hur väl tekniska indikatorer predikterar upprespektive nedgångar på aktiepriser. Vad gäller investeringsstrategierna återfanns inga tydliga samband mellan de utvalda indikatorerna och transaktionerna. Resultaten för andra delen av studien tyder på viss prediktiv förmåga för tekniska indikatorer på marknadsrörelser. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

Water monitoring is an essential process that managesthe well-being of freshwater ecosystems. However, it isgenerally an inefficient process as most data collection is donemanually. By combining wireless sensor technology and machinelearning techniques, projects such as iWater aim to modernizecurrent methods. The purpose of the iWater project is to developa network of smart sensors capable of collecting and analyzingwater quality-related data in real time.To contribute to this goal, a comparative study between theperformance of a centralized machine learning algorithm thatis currently used, and a distributed model based on a federatedlearning algorithm was done. The data used for training andtesting both models was collected by a wireless sensor developedby the iWater project. The centralized algorithm was used asthe basis for the developed distributed model. Due to lack ofsensors, the distributed model was simulated by down-samplingand dividing the sensor data into six data sets representing anindividual sensor. The results are similar for both models andthe developed algorithm reaches an accuracy of 98.41 %. / Vattenövervakning är en nödvändig processför att få inblick i sötvattensekosystems välmående. Dessvärreär det en kostsam och tidskrävande process då insamling avdata vanligen görs manuellt. Genom att kombinera trådlössensorteknologi och maskininlärnings algoritmer strävar projektsom iWater mot att modernisera befintliga metoder.Syftet med iWater är att skapa ett nätverk av smarta sensorersom kan samla in och analysera vattenkvalitetsrelaterade datai realtid. För att bidra till projektmålet görs en jämförandestudie mellan den prediktiva noggrannheten hos en centraliseradmaskininlärningsalgoritm, som i nuläget används, och endistribuerad modell baserad på federerat lärande. Data somanvänds för träning och testning av båda modellerna samladesin genom en trådlös sensor utvecklad inom iWater-projektet.Den centraliserade algoritmen användes som grund för denutvecklade distribuerade modellen. På grund av brist på sensorersimulerades den distribuerade modellen genom nedprovtagningoch uppdelning av data i sex datamängder som representerarenskilda sensorer. Resultaten för båda modellerna var liknandeoch den utvecklade algoritmen har en noggrannhet på 98.41 % / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

With the rapid increase of Internet of Things-devices(IoT), demand for new machine learning algorithms and modelshas risen. The focus of this project is implementing a federatedlearning (FL) algorithm to detect anomalies in measurementsmade by a water monitoring IoT-sensor. The FL algorithm trainsacross a collection of decentralized IoT-devices, each using thelocal data acquired from the specific sensor. The local machinelearning models are then uploaded to a mutual server andaggregated into a global model. The global model is sent back tothe sensors and is used as a template when training starts againlocally. In this project, we only have had access to one physicalsensor. This has forced us to virtually simulate sensors. Thesimulation was done by splitting the data gathered by the onlyexisting sensor. To deal with the long, sequential data gatheredby the sensor, a long short-term memory (LSTM) network wasused. This is a special type of artificial neural network (ANN)capable of learning long-term dependencies. After analyzing theobtained results it became clear that FL has the potential toproduce good results, provided that more physical sensors aredeployed. / I samband med den snabba ökningen avInternet of Things-enheter (IoT) har efterfrågan på nya algoritmeroch modeller för maskininlärning ökat. Detta projektfokuserar på att implementera en federated learning (FL) algoritmför att detektera avvikelser i mätdata från en sensorsom övervakar vattenkvaliteten. FL algoritmen tränar en samlingdecentraliserade IoT-enheter, var och en med hjälp av lokaldata från sensorn i fråga. De lokala maskininlärningsmodellernaladdas upp till en gemensam server och sammanställs till englobal modell. Den globala modellen skickas sedan tillbaka tillsensorerna och används som mall när den lokala träningen börjarigen. I det här projektet hade vi endast tillgång till en fysisksensor. Vi har därför varit tvungna att simulera sensorer. Dettagjordes genom att dela upp datamängden som samlats in frånden fysiska sensorn. För att hantera den långa sekventiella dataanvänds ett long short-term memory (LSTM) nätverk. Detta ären speciell typ av artificiellt neuronnät (ANN) som är kapabeltatt minnas mönster under en längre tid. Efter att ha analyseratresultaten blev det tydligt att FL har potentialen att produceragoda resultat, givet att fler fysiska sensorer implementeras. / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm