No description available.

Intelligent transportation systems (ITS) have garnered considerable attention for providing efficient traffic management solutions. Traffic flow forecasting is a crucial component of it which serves as the foundation for various state-of-the-art deep learning approaches. Initially, researchers recognized that significant temporal changes from traffic flow data for modelling. However, as researchers delved deeper into the underlying correlations within traffic flow data, they discovered that spatial information from the road network also plays a crucial role in accurate forecasting. Consequently, deep learning methods that incorporate Spatio-temporal representation have been employed to address traffic flow forecasting. Although recent solutions to this problem are impressive, it is essential to discuss the reasoning behind the architecture of the model. The expression of each feature relies on selecting appropriate models for feature extraction and designing architectures that minimize information loss during modeling. In this thesis, the work focuses on graph-based Spatio-temporal feature engineering. The experiments are divided into two parts: 1). explores the efficient architecture for expressing spatial-temporal information by considering both different sequential modelling approaches. 2). Based on the result obtained, the second experiment focuses on multi- scale modelling to capture informative Spatio-temporal feature. We propose a model that incorporates sequential modeling and captures multi-scale Spatiotemporal semantics by employing residual connections in different hierarchy. We validate our model using three datasets, each containing varying information for extraction. Taking into account the dataset characteristics and the model structure, our model outperforms the baselines and state-of-the-art models. The experimental results indicate that the performance of sequential modeling and multi-scale semantics, combined with thoughtful model design, significantly contribute to the overall forecasting performance. Furthermore, our work serves as inspiration for expressive data mining methods that rely on appropriate feature extraction models and architecture design, taking into consideration the information content within the dataset.

Effective information analysis generally boils down to the geometry of the data represented by a graph. Typical applications include social networks, transportation networks, the spread of epidemic disease, brain's neuronal networks, gene data on biological regulatory networks, telecommunication networks, knowledge graph, which are lying on the non-Euclidean graph domain. To describe the geometric structures, graph matrices such as adjacency matrix or graph Laplacian can be employed to reveal latent patterns. This thesis focuses on the theoretical analysis of graph neural networks and the development of methods for specific applications using graph representation. Four methods are proposed, including rational neural networks for jump graph signal estimation, RemezNet for robust attribute prediction in the graph, ICNet for integrated circuit security, and CNF-Net for dynamic circuit deobfuscation. For the first method, a recent important state-of-art method is the graph convolutional networks (GCN) nicely integrate local vertex features and graph topology in the spectral domain. However, current studies suffer from drawbacks: graph CNNs rely on Chebyshev polynomial approximation which results in oscillatory approximation at jump discontinuities since Chebyshev polynomials require degree $Omega$(poly(1/$epsilon$)) to approximate a jump signal such as $|x|$. To reduce complexity, RatioanlNet is proposed to integrate rational function and neural networks for graph node level embeddings. For the second method, we propose a method for function approximation which suffers from several drawbacks: non-robustness and infeasibility issue; neural networks are incapable of extracting analytical representation; there is no study reported to integrate the superiorities of neural network and Remez. This work proposes a novel neural network model to address the above issues. Specifically, our method utilizes the characterizations of Remez to design objective functions. To avoid the infeasibility issue and deal with the non-robustness, a set of constraints are imposed inspired by the equioscillation theorem of best rational approximation. The third method proposes an approach for circuit security. Circuit obfuscation is a recently proposed defense mechanism to protect digital integrated circuits (ICs) from reverse engineering. Estimating the deobfuscation runtime is a challenging task due to the complexity and heterogeneity of graph-structured circuit, and the unknown and sophisticated mechanisms of the attackers for deobfuscation. To address the above-mentioned challenges, this work proposes the first graph-based approach that predicts the deobfuscation runtime based on graph neural networks. The fourth method proposes a representation for dynamic size of circuit graph. By analyzing SAT attack method, a conjunctive normal form (CNF) bipartite graph is utilized to characterize the complexity of this SAT problem. To overcome the difficulty in capturing the dynamic size of the CNF graph, an energy-based kernel is proposed to aggregate dynamic features. / Doctor of Philosophy / Graph data is pervasive throughout most fields, including pandemic spread network, social network, transportation roads, internet, and chemical structure. Therefore, the applications modeled by graph benefit people's everyday life, and graph mining derives insightful opinions from this complex topology. This paper investigates an emerging technique called graph neural newton (GNNs), which is designed for graph data mining. There are two primary goals of this thesis paper: (1) understanding the GNNs in theory, and (2) apply GNNs for unexplored and values real-world scenarios. For the first goal, we investigate spectral theory and approximation theory, and a unified framework is proposed to summarize most GNNs. This direction provides a possibility that existing or newly proposed works can be compared, and the actual process can be measured. Specifically, this result demonstrates that most GNNs are either an approximation for a function of graph adjacency matrix or a function of eigenvalues. Different types of approximations are analyzed in terms of physical meaning, and the advantages and disadvantages are offered. Beyond that, we proposed a new optimization for a highly accurate but low efficient approximation. Evaluation of synthetic data proves its theoretical power, and the tests on two transportation networks show its potentials in real-world graphs. For the second goal, the circuit is selected as a novel application since it is crucial, but there are few works. Specifically, we focus on a security problem, a high-value real-world problem in industry companies such as Nvidia, Apple, AMD, etc. This problem is defined as a circuit graph as apply GNN to learn the representation regarding the prediction target such as attach runtime. Experiment on several benchmark circuits shows its superiority on effectiveness and efficacy compared with competitive baselines. This paper provides exploration in theory and application with GNNs, which shows a promising direction for graph mining tasks. Its potentials also provide a wide range of innovations in graph-based problems.

In the development process of novel cancer drugs, one important aspect is to identify patient populations with a high risk of early death so that resources can be focused on patients with the highest medical unmet need. Many cancer types are heterogeneous and there is a need to identify patients with aggressive diseases, meaning a high risk of early death, compared to patients with indolent diseases, meaning a low risk of early death. Predictive modeling can be a useful tool for risk stratification in clinical practice, enabling healthcare providers to treat high-risk patients early and progressively, while applying a less aggressive watch-and-wait strategy for patients with a lower risk of death. This is important from a clinical perspective, but also a health economic perspective since society has limited resources, and costly drugs should be given to patients that can benefit the most from a specific treatment. Thus, the goal of predictive modeling is to ensure that the right patient will have access to the right drug at the right time. In the era of personalized medicine, Artificial Intelligence (AI) applied to high-quality data will most likely play an important role and many techniques have been developed. In particular, Graph Neural Network (GNN) is a promising tool since it captures the complexity of high dimensional data modeled as a graph. In this work, we have applied Network Representation Learning (NRL) techniques to predict survival, using pseudonymized patient-level data from national health registries in Sweden. Over the last decade, more health data of increased complexity has become available for research, and therefore precision medicine could take advantage of this trend by bringing better healthcare to the patients. However, it is important to develop reliable prediction models that not only show high performances but take into consideration privacy, avoiding any leakage of personal information. The present study contributes novel insights related to GNN performance in different survival prediction tasks, using population-based unique nationwide data. Furthermore, we also explored how privacy methods impact the performance of the models when applied to the same dataset. We conducted a set of experiments across 6 dataset using 8 models measuring both AUC, Precision and Recall. Our evaluation results show that Graph Neural Networks were able to reach accuracy performance close to the models used in clinical practice and constantly outperformed, by at least 4.5%, the traditional machine learning methods. Furthermore, the study demonstrated how graph modeling, when applied based on knowledge from clinical experts, performed well and showed high resiliency to the noise introduced for privacy preservation. / I utvecklingsprocessen för nya cancerläkemedel är en viktig aspekt att identifiera patientgrupper med hög risk för tidig död, så att resurser kan fokuseras på patientgrupper med störst medicinskt behov. Många cancertyper är heterogena och det finns ett behov av att identifiera patienter med aggressiv sjukdom, vilket innebär en hög risk för tidig död, jämfört med patienter med indolenta sjukdom, vilket innebär lägre risk för tidig död. Prediktiv modellering kan vara ett användbart verktyg för riskstratifiering i klinisk praxis, vilket gör det möjligt för vårdgivare att behandla patienter olika utifrån individuella behov. Detta är viktigt ur ett kliniskt perspektiv, men också ur ett hälsoekonomiskt perspektiv eftersom samhället har begränsade resurser och kostsamma läkemedel bör ges till de patienter som har störst nytta av en viss behandling. Målet med prediktiv modellering är således att möjliggöra att rätt patient får tillgång till rätt läkemedel vid rätt tidpunkt. Framför allt är Graph Neural Network (GNN) ett lovande verktyg eftersom det fångar komplexiteten hos högdimensionella data som modelleras som ett diagram. I detta arbete har vi tillämpat tekniker för inlärning av grafrepresentationer för att prediktera överlevnad med hjälp av pseudonymiserade data från nationella hälsoregister i Sverige. Under det senaste decennierna har mer hälsodata av ökad komplexitet blivit tillgänglig för forskning. Även om denna ökning kan bidra till utvecklingen av precisionsmedicinen är det viktigt att utveckla tillförlitliga prediktionsmodeller som tar hänsyn till patienters integritet och datasäkerhet. Den här studien kommer att bidra med nya insikter om GNNs prestanda i prediktiva överlevnadsmodeller, med hjälp av populations -baserade data. Dessutom har vi också undersökt hur integritetsmetoder påverkar modellernas prestanda när de tillämpas på samma dataset. Sammanfattningsvis, Graph Neural Network kan uppnå noggrannhets -prestanda som ligger nära de modeller som tidigare använts i klinisk praxis och i denna studie preserade de alltid bättre än traditionella maskininlärnings -metoder. Studien visisade vidare hur grafmodellering som utförs i samarbete med kliniska experter kan vara effektiva mot det brus som införs av olika integritetsskyddstekniker.

Indiana University-Purdue University Indianapolis (IUPUI) / Many complex processes can be viewed as sequential events on a network. In this thesis, we study the interplay between a network and the event sequences on it. We first focus on predicting events on a known network. Examples of such include: modeling retweet cascades, forecasting earthquakes, and tracing the source of a pandemic. In specific, given the network structure, we solve two types of problems - (1) forecasting future events based on the historical events, and (2) identifying the initial event(s) based on some later observations of the dynamics. The inverse problem of inferring the unknown network topology or links, based on the events, is also of great important. Examples along this line include: constructing influence networks among Twitter users from their tweets, soliciting new members to join an event based on their participation history, and recommending positions for job seekers according to their work experience. Following this direction, we study two types of problems - (1) recovering influence networks, and (2) predicting links between a node and a group of nodes, from event sequences.

In this work, we leverage the recent developments in computer science to address several of the challenges in current literature-based discovery (LBD) solutions. First, LBD solutions cannot use semantics or are too computational complex. To solve the problems we propose a generative model OverlapLDA based on topic modeling, which has been shown both effective and efficient in extracting semantics from a corpus. We also introduce an inference method of OverlapLDA. We conduct extensive experiments to show the effectiveness and efficiency of OverlapLDA in LBD. Second, we expand LBD to a more complex and realistic setting. The settings are that there can be more than one concept connecting the input concepts, and the connectivity pattern between concepts can also be more complex than a chain. Current LBD solutions can hardly complete the LBD task in the new setting. We simplify the hypotheses as concept sets and propose LBDSetNet based on graph neural networks to solve this problem. We also introduce different training schemes based on self-supervised learning to train LBDSetNet without relying on comprehensive labeled hypotheses that are extremely costly to get. Our comprehensive experiments show that LBDSetNet outperforms strong baselines on simple hypotheses and addresses complex hypotheses.

Social networks can be helpful for the analysis of behaviour of people. An existing social network is rarely available, and its nodes and edges have to be inferred from not necessarily graph data. Link prediction can be used to either correct inaccuracies or to forecast links about to appear in the future. In this work, we study the prediction of miss- ing links in a social network inferred from real-world bank data. We review and compare both verified and modern approaches to link prediction. Following the advancements of deep learning in recent years, we primarily focus on graph neural networks, and their ability to scale to large networks. We propose an adjustment to an existing graph neural network method and show that its performance is either comparable with or outperform- ing the original method. The comparison is performed on two social networks inferred from the same data. We show that it is relatively hard to outperform the verified link prediction methods with graph neural networks. 1

This report is done in collaboration with WirelessCar for the master of science thesis at Halmstad University. Many different parameters influence fuel consumption. The objective of the report is to evaluate if Graph neural networks are a practical model to perform fuel consumption prediction on areas. The model uses a partitioning of geographical locations of trip observations to capture their spatial information. The project also proposes a method to capture the non-stationary behavior of vehicles by defining a vehicle node as a separate entity. The model then captures their different features in a dense layer neural network and utilizes message passing to capture context about neighboring nodes. The model is compared to a baseline neural network with a similar network architecture as the graph neural network. The data is partitioned to define an area with Kmeans and static gridnet partition with and without terrain details. This partition is used to structure a homogeneous graph that is underperforming. The practical drawbacks of the initial homogeneous graph are inspected and addressed to develop a heterogeneous graph that can outperform the neural network baseline.

No description available.

<p dir="ltr">This thesis presents two studies in search for violation of lepton flavor universality as predicted by the Standard Model. The first searches for signs of the violation by studying beyond the Standard Model (BSM) physics models involving same flavor and opposite sign dilepton pair and bottom quarks as final states. This study was done using the dataset collected during years of 2016, 2017 and 2018, with center of mass energy $\sqrt{s} = 13$ TeV and integrated luminosity of 138 $fb^-1$. In the study, scale factors were derived in order to correct deviations between simulation and real life data, specifically for high transverse momentum muons and top\&anti-top quark background. Furthermore, lower limits of energy scale were calculated leading to exclusion of the BSM models with energy scale values lower than that of the calculated value.</p><p dir="ltr">The second study also searches for of the lepton flavor universality violation, but in the specific decay of a tauon into three muons ($\tau \rightarrow 3\mu$). In the study, graph based neural network model (GNN) designed to classify $\tau \rightarrow 3\mu$ events at the CMS detector was converted to high level synthesis (HLS) code, so that the GNN could be coded into a custom hardware such as field programmable gate arrays (FPGA) for deployment. Moreover, techniques such as pruning and quantization were applied in an attempt to make the GNN more light weight, due to strict requirements of FPGA.</p>