New regulations are placed on Swedish Telecom Service Providers (TSPs) due to a rising concern for safeguarding network security and privacy in the face of ever-evolving cyber threats. These regulations demand that Swedish telecom companies expand their data security strategies with proactive security measures. Logs, serving as digital footprints in IT infrastructure, play a crucial role in identifying anomalies that could indicate security breaches. Deep Learning (DL) has been used to detect anomalies in logs due to its ability to discern intricate patterns within the data. By leveraging deep learning-based models, it is not only possible to identify anomalies but also to predict and mitigate potential threats within the telecom network. An LSTM autoencoder was implemented to detect anomalies in two separate multivariate temporal log datasets; the BETH cybersecurity dataset, and a Cisco log dataset that was created specifically for this thesis. The empirical results in this thesis show that the LSTM autoencoder reached an ROC AUC of 99.5% for the BETH dataset and 76.6% for the Cisco audit dataset. The use of an additional anomaly detection aid in the Cisco audit dataset let the model reach an ROC AUC of 99.6%. The conclusion that could be drawn from this work was that the systematic approach to developing a deep learning model for anomaly detection in log data was efficient. However, the study’s findings raise crucial considerations regarding the appropriateness of various log data for deep learning models used in anomaly detection. / Nya föreskrifter har införts för svenska telekomtjänsteleverantörer på grund av en ökad angelägenhet av att säkerställa nätverkssäkerhet och integritet inför ständigt föränderliga cyberhot. Dessa föreskrifter kräver att svenska telekomföretag utvidgar sina dataskyddsstrategier med proaktiva säkerhetsåtgärder. Loggar, som fungerar som digitala fotspår inom IT-infrastruktur, spelar en avgörande roll för att identifiera avvikelser som kan tyda på säkerhetsintrång. Djupinlärning har använts för att upptäcka avvikelser i loggar på grund av dess förmåga att urskilja intrikata mönster inom data. Genom att utnyttja modeller baserade på djupinlärning är det inte bara möjligt att identifiera avvikelser utan även att förutsäga samt mildra konsekvenserna av potentiella hot inom telekomnätet. En LSTM-autoencoder implementerades för att upptäcka avvikelser i två separata multivariata tidsserielogguppsättningar; BETH-cybersäkerhetsdatauppsättningen och en Cisco-loggdatauppsättning som skapades specifikt för detta arbete. De empiriska resultaten i denna avhandling visar att LSTM-autoencodern uppnådde en ROC AUC på 99.5% för BETH-datauppsättningen och 76.6% för Cisco-datauppsättningen. Användningen av ett ytterligare avvikelsedetekteringsstöd i Cisco-datauppsättningen möjliggjorde att modellen uppnådde en ROC AUC på 99.6%. Slutsatsen som kunde dras från detta arbete var att den systematiska metoden för att utveckla en djupinlärningsmodell för avvikelsedetektering i loggdata var effektiv. Dock väcker studiens resultat kritiska överväganden angående lämpligheten av olika loggdata för djupinlärningsmodeller som används för avvikelsedetektering.

Financial trading data presents a unique challenge for anomaly detection due to its high dimensionality and often lack of labelled anomalous examples. Nevertheless, it is of great interest for financial institutions to gain insight into potential trading activities that might lead to financial losses and reputational damage. Given the complexity and unlabelled nature of this financial data, deep learning models such as the Transformer model are particularly suited for this task.   This work investigates the application of a Transformer-based autoencoder for anomaly detection in unlabelled financial transaction data with sequential characteristics. To assess the model's ability to detect anomalies and analyse the effects of class imbalance, synthetic anomalies are injected into the dataset. This creates a controlled environment where the model's performance can be evaluated but also the affects of imbalance can be investigated.    Two approaches are particularly explored for anomaly detection purposes: an unsupervised approach and a semi-supervised approach that explicitly leverages the presence of anomalies in the training data. Experiments suggest that while the unsupervised approach can detect anomalies with distinctive features, its performance suffers when anomalies are included in the training data since the model tends to reconstruct them. Conversely, the semi-supervised approach effectively addresses this limitation, demonstrating a clear advantage in the presence of class imbalance. While synthetic anomalies enable controlled evaluation and class imbalance analysis, generalizability to real-world financial data requires true anomalies.

<div> <p>Sound and vibration analysis is a prominent tool used for scientific investigations in various fields such as structural model identification or dynamic behavior studies. In manufacturing fields, the vibration signals collected through commercial sensors are utilized to monitor machine health, for sustainable and cost-effective manufacturing.</p> <p> Recently, the development of commercial sensors and computing environments have encouraged researchers to combine gathered data and Machine Learning (ML) techniques, which have been proven to be efficient for categorical classification problems. These discriminative algorithms have been successfully implemented in monitoring problems in factories, by simulating faulty situations. However, it is difficult to identify all the sources of anomalies in a real environment. </p> <p>In this paper, a Neural Network (NN) application on a KUKA KR6 robot arm is introduced, as a solution for the limitations described above. Specifically, the autoencoder architecture was implemented for anomaly detection, which does not require the predefinition of faulty signals in the training process. In addition, stethoscopes were utilized as alternative sensing tools as they are easy to handle, and they provide a cost-effective monitoring solution. To simulate the normal and abnormal conditions, different load levels were assigned at the end of the robot arm according to the load capacity. Sound signals were recorded from joints of the robot arm, then meaningful features were extracted from spectrograms of the sound signals. The features were utilized to train and test autoencoders. During the autoencoder process, reconstruction errors (REs) between the autoencoder’s input and output were computed. Since autoencoders were trained only with features corresponding to normal conditions, RE values corresponding to abnormal features tend to be higher than those of normal features. In each autoencoder, distributions of the RE values were compared to set a threshold, which distinguishes abnormal states from the normal states. As a result, it is suggested that the threshold of RE values can be utilized to determine the condition of the robot arm.</p> </div> <br>

Gene expression analysis provides genome-wide insights into the transcriptional activity of a cell. One of the first computational steps in exploration and analysis of the gene expression data is clustering. With a number of standard clustering methods routinely used, most of the methods do not take prior biological information into account. Here, we propose a new approach for gene expression clustering analysis. The approach benefits from a new deep learning architecture, Robust Autoencoder, which provides a more accurate high-level representation of the feature sets, and from incorporating prior system-wide biological information into the clustering process. We tested our approach on two gene expression datasets and compared the performance with two widely used clustering methods, hierarchical clustering and k-means, and with a recent deep learning clustering approach. Our approach outperformed all other clustering methods on the labeled yeast gene expression dataset. Furthermore, we showed that it is better in identifying the functionally common clusters than k-means on the unlabeled human gene expression dataset. The results demonstrate that our new deep learning architecture can generalize well the specific properties of gene expression profiles. Furthermore, the results confirm our hypothesis that the prior biological network knowledge is helpful in the gene expression clustering.

Deep learning is usually applied to static datasets. If used for classification based on data streams, it is not easy to take into account a non-stationarity. This thesis presents work in progress on a new method for online deep classifi- cation learning in data streams with slow or moderate drift, highly relevant for the application domain of malware detection. The method uses a combination of multilayer perceptron and variational autoencoder to achieve constant mem- ory consumption by encoding past data to a generative model. This can make online learning of neural networks more accessible for independent adaptive sys- tems with limited memory. First results for real-world malware stream data are presented, and they look promising. 1

Predictive Maintenance (PM) has been increasingly adopted in the Automotive industry, in the recent decades along with conventional approaches such as the Preventive Maintenance and Diagnostic/Corrective Maintenance, since it provides many advantages to estimate the failure before the actual occurrence proactively, and also being adaptive to the present status of the vehicle, in turn allowing flexible maintenance schedules for efficient repair or replacing of faulty components. PM necessitates the storage and analysis of large amounts of sensor data. This requirement can be a challenge in deploying this method on-board the vehicles due to the limited storage and computational power on the hardware of the vehicle. Hence, this thesis seeks to obtain low dimensional descriptive features from high dimensional data using Representation Learning. This low dimensional representation will be used for predicting vehicle faults, specifically Turbocharger related failures. Since the Logged Vehicle Data (LVD) was base on all the data utilized in this thesis, it allowed for the evaluation of large populations of trucks without requiring additional measuring devices and facilities. The gradual degradation methodology is considered for describing vehicle condition, which allows for modeling the malfunction/ failure as a continuous process rather than a discrete flip from healthy to an unhealthy state. This approach eliminates the challenge of data imbalance of healthy and unhealthy samples. Two important hypotheses are presented. Firstly, Parallel StackedClassical Autoencoders would produce better representations com-pared to individual Autoencoders. Secondly, employing Learned Em-beddings on Categorical Variables would improve the performance of the Dimensionality reduction. Based on these hypotheses, a model architecture is proposed and is developed on the LVD. The model is shown to achieve good performance, and in close standards to the previous state-of-the-art research. This thesis, finally, illustrates the potential to apply parallel stacked architectures with Learned Embeddings for the Categorical features, and a combination of feature selection and extraction for numerical features, to predict the Remaining Useful Life (RUL) of a vehicle, in the context of the Turbocharger. A performance improvement of 21.68% with respect to the Mean Absolute Error (MAE) loss with an 80.42% reduction in the size of data was observed.

Wireless communications technologies have experienced an exponential development during the last decades. 5G is a prominent exponent whose one of its crucial component is the Massive MIMO technology. By supporting multiple streams of signals it allows a revamped signal reconstruction in terms of mobile traffic size, data rate, latency, and reliability. In this thesis work, we isolated this technology into a SIMOapproach (Single-Input Multiple-Output) to explore a Machine Learning modeling to address the so-called Channel Prediction problem. Generally, the algorithms available to perform Channel Estimation in FDD and TDD deployments incur computational complexity downsides and require explicit feedback from client devices, which is typically prohibitive. This thesis work focuses on Channel Prediction by aims of employing Machine and deep Learning models in order to reduce the computational complexity by further relying in statistical modeling/learning. We explored the cross-Frequency Subband prediction intra-TTI (Transmission Time Interval) by means of proposing 3 three models. These intended to leverage frequency Multipath Components dependencies along TTIs. The first two ones are Probabilistic Principal Components Analysis (PPCA) and its Bayesiancounterpart, Bayesian Principal Components Analysis (BPCA). Then, we implemented Deep Learning Variational Encoder-Decoder (VED) architecture. These three models are intended to deal with the hugely high-dimensional space of the 4 datasets used by its intrinsic dimensionality reduction. The PPCA method was on average five times better than the VED alternative in terms of MSE accounting for all the datasets used.

<p>Queueing systems are everywhere: in transportation networks, service centers, communication systems, clinics, manufacturing systems, etc. In this dissertation, we contribute to the theory of queueing in two aspects. In the first part, we dilate the interplay between retrials and strategic arrival behavior in single-class queueing networks. Specifically, we study a variation of the ‘Network Concert Queueing Game,’ wherein a fixed but large number of strategic users arrive at a network of queues where they can be routed to other queues in the network following a fixed routing matrix, or potentially fedback to the end of the queue they arrive at. Working in a non-atomic setting, we prove the existence of Nash equilibrium arrival and routing profiles in three simple, but non-trivial, network topologies/architectures. In two of them, we also prove the uniqueness of the equilibrium. Our results prove that Nash equilibrium decisions on when to arrive and which queue to join in a network are substantially impacted by routing, inducing ‘herding’ behavior under certain conditions on the network architecture. Our theory raises important design implications for capacity-sharing in systems with strategic users, such as ride-sharing and crowdsourcing platforms.</p> <p><br></p> <p>In the second part, we develop a new method of data-driven model calibration or estimation for queueing models. Statistical and theoretical analyses of traffic traces show that the doubly stochastic Poisson processes are appropriate models of high intensity traffic arriving at an array of service systems. On the other hand, the statistical estimation of the underlying latent stochastic intensity process driving the traffic model involves a rather complicated nonlinear filtering problem. In this thesis we use deep neural networks to ‘parameterize’ the path measures induced by the stochastic intensity process, and solve this nonlinear filtering problem by maximizing a tight surrogate objective called the evidence lower bound (ELBO). This framework is flexible in the sense that we can also estimate other stochastic processes (e.g., the queue length process) and their related parameters (e.g., the service time distribution). We demonstrate the effectiveness of our results through extensive simulations. We also provide approximation guarantees for the estimation/calibration problem. Working with the Markov chain induced by the Euler-Maruyama discretization of the latent diffusion, we show that (1) there exists a sequence of approximate data generating distributions that converges to the “ground truth” distribution in total variation distance; (2) the variational gap is strictly positive for the optimal solution to the ELBO. Extending to the non-Markov setting, we identify the variational gap minimizing approximate posterior for an arbitrary (known) posterior and further, prove a lower bound on the optimal ELBO. Recent theoretical results on optimizing the ELBO for related (but ultimately different) models show that when the data generating distribution equals the ground truth distribution and the variational gap is zero, the probability measures that achieve these conditions also maximize the ELBO. Our results show that this may not be true in all problem settings.</p>

The most common approach to analysing satellite imagery is building or object segmentation,which expects an algorithm to find and segment objects with specific boundaries thatare present in the satellite imagery. The company Vricon takes satellite imagery analysisfurther with the goal of reproducing the entire world into a 3D mesh. This 3D reconstructionis performed by a set of complex algorithms excelling in different object reconstructionswhich need sufficient labeling in the original 2D satellite imagery to ensure validtransformations. Vricon believes that the labeling of areas can be used to improve the algorithmselection process further. Therefore, the company wants to investigate if multinomiallarge area classification can be performed successfully using the satellite image data availableat the company. To enable this type of classification, the company’s gold-standarddataset containing labeled objects such as individual buildings, single trees, roads amongothers, has been transformed into an large area gold-standard dataset in an unsupervisedmanner. This dataset was later used to evaluate large area classification using several stateof-the-art Deep Convolutional Neural Network (DCNN) semantic segmentation architectureson both RGB as well as RGB and Digital Surface Model (DSM) height data. Theresults yield close to 63% mIoU and close to 80% pixel accuracy on validation data withoutusing the DSM height data in the process. This thesis additionally contributes with a novelapproach for large area gold-standard creation from existing object labeled datasets.

京都大学 / 新制・課程博士 / 博士(情報学) / 甲第23540号 / 情博第770号 / 新制||情||131(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)准教授 吉井 和佳, 教授 河原 達也, 教授 西野 恒, 教授 鹿島 久嗣 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM