Identifying new potential treatment options for medical conditions that cause human disease burden is a central task of biomedical research. Since all candidate drugs cannot be tested with animal and clinical trials, in vitro approaches are first attempted to identify promising candidates. Likewise, identifying other essential relations (e.g., causation, prevention) between biomedical entities is also critical to understand biomedical processes. Hence, it is crucial to develop automated relation prediction systems that can yield plausible biomedical relations to expedite the discovery process. In this dissertation, we demonstrate three approaches to predict treatment relations between biomedical entities for the drug repositioning task using existing biomedical knowledge bases. Our approaches can be broadly labeled as link prediction or knowledge base completion in computer science literature. Specifically, first we investigate the predictive power of graph paths connecting entities in the publicly available biomedical knowledge base, SemMedDB (the entities and relations constitute a large knowledge graph as a whole). To that end, we build logistic regression models utilizing semantic graph pattern features extracted from the SemMedDB to predict treatment and causative relations in Unified Medical Language System (UMLS) Metathesaurus. Second, we study matrix and tensor factorization algorithms for predicting drug repositioning pairs in repoDB, a general purpose gold standard database of approved and failed drug–disease indications. The idea here is to predict repoDB pairs by approximating the given input matrix/tensor structure where the value of a cell represents the existence of a relation coming from SemMedDB and UMLS knowledge bases. The essential goal is to predict the test pairs that have a blank cell in the input matrix/tensor based on the shared biomedical context among existing non-blank cells. Our final approach involves graph convolutional neural networks where entities and relation types are embedded in a vector space involving neighborhood information. Basically, we minimize an objective function to guide our model to concept/relation embeddings such that distance scores for positive relation pairs are lower than those for the negative ones. Overall, our results demonstrate that recent link prediction methods applied to automatically curated, and hence imprecise, knowledge bases can nevertheless result in high accuracy drug candidate prediction with appropriate configuration of both the methods and datasets used.

In this thesis, an anomaly detection framework is assessed and fine-tuned to detect and explain anomalies in a power station, where EtherCAT, an Industrial Control System, is employed for monitoring. The chosen framework is based on a previously published Graph Neural Network (GNN) model, utilizing attention mechanisms to capture complex relationships between diverse measurements within the EtherCAT system. To address the challenges in graph learning and improve model performance and computational efficiency, the study introduces a novel similarity thresholding approach. This approach dynamically selects the number of neighbors for each node based on their similarity instead of adhering to a fixed 'k' value, thus making the learning process more adaptive and efficient. Further in the exploration, the study integrates Extreme Value Theory (EVT) into the framework to set the anomaly detection threshold and assess its effectiveness. The effect of temporal features on model performance is examined, and the role of seconds of the day as a temporal feature is notably highlighted. These various methodological innovations aim to refine the application of the attention based GNN framework to the EtherCAT system. The results obtained in this study illustrate that the similarity thresholding approach significantly improves the model's F1 score compared to the standard TopK approach. The inclusion of seconds of the day as a temporal feature led to modest improvements in model performance, and the application of EVT as a thresholding technique was explored, although it did not yield significant benefits in this context. Despite the limitations, including the utilization of a single-day dataset for training, the thesis provides valuable insights for the detection of anomalies in EtherCAT systems, contributing both to the literature and the practitioners in the field. It lays the groundwork for future research in this domain, highlighting key areas for further exploration such as larger datasets, alternative anomaly detection techniques, and the application of the framework in streaming data environments. / I denna avhandling utvärderas och finslipas ett ramverk för att detektera och förklara anomalier på ett kraftverk, där EtherCAT, ett industriellt styrsystem, används för övervakning. Det valda ramverket är baserat på en tidigare publicerad graf neurala nätverksmodell (GNN) som använder uppmärksamhetsmekanismer för att fånga komplexa samband mellan olika mätningar inom EtherCAT-systemet. För att hantera utmaningar inom grafiskt lärande och förbättra modellens prestanda och beräkningseffektivitet introducerar studien en ny metod för likhetsgränsdragning. Denna metod väljer dynamiskt antalet grannar för varje nod baserat på deras likhet istället för att hålla sig till ett fast 'k'-värde, vilket gör inlärningsprocessen mer anpassningsbar och effektiv. I en vidare undersökning integrerar studien extremvärdesteori (EVT) i ramverket för att sätta tröskeln för detektering av anomalier och utvärdera dess effektivitet. Effekten av tidsberoende egenskaper på modellens prestanda undersöks, och sekunder av dagen som en tidsberoende egenskap framhävs särskilt. Dessa olika metodologiska innovationer syftar till att förädla användningen av det uppmärksamhetsbaserade GNN-ramverket på EtherCAT-systemet. Resultaten som erhållits i denna studie illustrerar att likhetsgränsdragning väsentligt förbättrar modellens F1-poäng jämfört med den standardiserade TopK-metoden. Inkluderingen av sekunder av dagen som en tidsberoende egenskap ledde till blygsamma förbättringar i modellens prestanda, och användningen av EVT som en tröskelmetod undersöktes, även om den inte gav några betydande fördelar i detta sammanhang. Trots begränsningarna, inklusive användningen av ett dataset för endast en dag för träning, ger avhandlingen värdefulla insikter för detektering av anomalier i EtherCAT-system, och bidrar både till litteraturen och praktiker inom området. Den lägger grunden för framtida forskning inom detta område, och belyser nyckelområden för ytterligare utforskning såsom större dataset, alternativa tekniker för detektering av anomalier och tillämpningen av ramverket i strömmande data-miljöer.

Many novel graph neural network models have reported an impressive performance on benchmark dataset, but the theory behind these networks is still being developed. In this thesis, we study the trajectory of Gradient descent (GD) and Stochastic gradient descent (SGD) in the loss landscape of Graph neural networks by replicating Xing et al. [1] study for feed-forward networks. Furthermore, we empirically examine if the training process could be accelerated by an optimization algorithm inspired from Stochastic gradient Langevin dynamics and what effect the topology of the graph has on the convergence of GD by perturbing its structure. We find that the loss landscape is relatively flat and that SGD does not encounter any significant obstacles during its propagation. The noise-induced gradient appears to aid SGD in finding a stationary point with desirable generalisation capabilities when the learning rate is poorly optimized. Additionally, we observe that the topological structure of the graph plays a part in the convergence of GD but further research is required to understand how. / Många nya grafneurala nätverk har visat imponerande resultat på existerande dataset, dock är teorin bakom dessa nätverk fortfarande under utveckling. I denna uppsats studerar vi banor av gradientmetoden (GD) och den stokastiska gradientmetoden (SGD) i lösningslandskapet till grafiska faltningsnätverk genom att replikera studien av feed-forward nätverk av Xing et al. [1]. Dessutom undersöker vi empiriskt om träningsprocessen kan accelereras genom en optimeringsalgoritm inspirerad av Stokastisk gradient Langevin dynamik, samt om grafens topologi har en inverkan på konvergensen av GD genom att ändra strukturen. Vi ser att lösningslandskapet är relativt plant och att bruset inducerat i gradienten verkar hjälpa SGD att finna stabila stationära punkter med önskvärda generaliseringsegenskaper när inlärningsparametern har blivit olämpligt optimerad. Dessutom observerar vi att den topologiska grafstrukturen påverkar konvergensen av GD, men det behövs mer forskning för att förstå hur.

<p dir="ltr">This research involves numerous network intrusion techniques through novel applications of graph representation learning and image representation learning. The methods are tested on multiple publicly available network flow datasets.</p>

Les relations entre la structure des composés chimiques et leurs propriétés sont complexes et à haute dimension. Dans le processus de développement de médicaments, plusieurs proprié- tés d’un composé doivent souvent être optimisées simultanément, ce qui complique encore la tâche. Ce travail explore deux représentations des composés chimiques pour les tâches de prédiction des propriétés. L’objectif de ces représentations proposées est d’améliorer l’explicabilité afin de faciliter le processus d’optimisation des propriétés des composés. Pre- mièrement, nous décomposons l’algorithme ECFP (Extended connectivity Fingerprint) et le rendons plus simple pour la compréhension humaine. Nous remplaçons une fonction de hachage sujet aux collisions par une relation univoque de sous structure à bit. Nous consta- tons que ce changement ne se traduit pas par une meilleure performance prédictive d’un perceptron multicouche par rapport à l’ECFP. Toutefois, si la capacité du prédicteur est ra- menée à celle d’un prédicteur linéaire, ses performances sont meilleures que celles de l’ECFP. Deuxièmement, nous appliquons l’apprentissage automatique à l’analyse des paires molécu- laires appariées (MMPA), un paradigme de conception du développement de médicaments. La MMPA compare des paires de composés très similaires, dont la structure diffère par une modification sur un site. Nous formons des modèles de prédiction sur des paires de com- posés afin de prédire les différences d’activité. Nous utilisons des contraintes de similarité par paires comme MMPA, mais nous utilisons également des paires échantillonnées de façon aléatoire pour entraîner les modèles. Nous constatons que les modèles sont plus performants sur des paires choisies au hasard que sur des paires avec des contraintes de similarité strictes. Cependant, les meilleurs modèles par paires ne sont pas capables de battre les performances de prédiction du modèle simple de base. Ces deux études, RCFP et comparaisons par paires, visent à aborder la prédiction des propriétés d’une manière plus compréhensible. En utili- sant l’intuition et l’expérience des chimistes médicinaux dans le cadre de la modélisation prédictive, nous espérons encourager l’explicabilité en tant que composante nécessaire des modèles cheminformatiques prédictifs. / The relationships between the structure of chemical compounds and their properties are complex and high dimensional. In the drug development process, multiple properties of a compound often need to be optimized simultaneously, further complicating the task. This work explores two representations of chemical compounds for property prediction tasks. The goal of these suggested representations is improved explainability to better understand the compound property optimization process. First, we decompose the Extended Connectivity Fingerprint (ECFP) algorithm and make it more straightforward for human understanding. We replace a collision-prone hash function with a one-to-one substructure-to-bit relationship. We find that this change which does not translate to higher predictive performance of a multi- layer perceptron compared to ECFP. However, if the capacity of the predictor is lowered to that of a linear predictor, it does perform better than ECFP. Second, we apply machine learning to Matched Molecular Pair Analysis (MMPA), a drug development design paradigm. MMPA compares pairs of highly similar compounds, differing in structure by modification at one site. We train prediction models on pairs of compounds to predict differences in activity. We use pairwise similarity constraints like MMPA, but also use randomly sampled pairs to train the models. We find that models perform better on randomly chosen pairs than on pairs with strict similarity constraints. However, the best pairwise models are not able to beat the prediction performance of the simpler baseline single model. Both of these investigations, RCFP and pairwise comparisons, aim to approach property prediction in a more explainable way. By using intuition and experience of medicinal chemists within predictive modelling, we hope to encourage explainability as a necessary component of predictive cheminformatic models.