<p>    </p> <p>The counting process is the fundamental of many real-world problems with event data. Poisson process, used as the background intensity of Hawkes process, is the most commonly used point process. The Hawkes process, a self-exciting point process fits to temporal event data, spatial-temporal event data, and event data with covariates. We study the Hawkes process that fits to heterogeneous drug overdose data via a novel semi-parametric approach. The counting process is also related to survival data based on the fact that they both study the occurrences of events over time. We fit a Cox model to temporal event data with a large corpus that is processed into high dimensional covariates. We study the significant features that influence the intensity of events. </p>

<p dir="ltr">Explainable AI (XAI) is a class of post-hoc analysis tools, which include a large selection of algorithms developed to increase transparency in the decision-making process of Machine Learning (ML) models. These tools aim to provide users with interpretations of the data and the model. However, despite the abundance of options and their potential in identifying and decomposing model behavior, XAI's inability to quantitatively assess trustworthiness, due to the lack of quantifiable metrics, has resulted in low adoption in real-world applications. In contrast, traditional methods to evaluate trust such as uncertainty quantification, robust testing, and user studies scale well with large models and datasets, thanks to their reliance on quantifiable metrics. However, they do not offer the same level of transparency and qualitative assessments as XAI to make the models more interpretable, which are a key component of the multi-faceted trustworthiness assessment.</p><p dir="ltr">To bridge this gap, I propose a framework in which explanations produced by XAI are ensembled across a portfolio of models. These ensembled explanations are then used for both quantitative and qualitative comparison to evaluate trust in the models. The goal is to leverage these explanations to assess trustworthiness driven by transparency. The framework also identifies areas of consensus or disagreement among the ensembled explanations. Further leverage the presence or absence of consensus to bin model reasoning to indicate weaknesses, misalignment to user expectations, and/or distribution shifts.</p><p dir="ltr">A preliminary investigation of the proposed framework is carried out on multivariate time-series data from NASA's Commercial Modular Aero-Propulsion System Simulation (CMAPSS) to model and predict turbojet engine degradation. This approach uses three distinct ML models to forecast the remaining useful life (RUL) of the engine. Using the proposed framework, influential system parameters contributing to engine degradation in each model are identified via XAI. These explanations are ensembled and compared to assess consensus. Ultimately, the models disagree on the extent of certain features contributing to the failure. However, experimental literature supports this finding as modeling engine degradation can be sensitive to the type of failure mode. Additionally, certain model architectures work better for certain types of data patterns, leading to recommendations on expert models. With these results and understanding of the intricacies of the framework, it is revised and implemented on a more complex application with a different data type and task: defect detection in robotic manipulation. The ARMBench (Amazon Robotic Manipulation Benchmark) dataset is used to train computer vision models for an image-based multi-classification problem and explained using activation maps. In this use case, both upstream and downstream influences and benefits of the framework are assessed while assessing the trustworthiness of the model and its predictions. The framework throws light on the strengths and weaknesses of the models, dataset, and deployment. Aiding in identifying strategies to mitigate weak and untrustworthy models. </p>

<p dir="ltr">This dissertation introduces a novel graph embedding paradigm, leveraging a force-directed scheme for graph embedding. In the field of graph embedding, an "embedding" refers to the process of transforming elements of a graph such as nodes, or edges, or potentially other structural information of the graph into a low-dimensional space, typically a vector space, while preserving the graph's structural properties as much as possible. The dimensions of the space are supposed to be much smaller than the elements of the graph that are to be embedded. This transformation results in a set of vectors, with each vector representing a node (or edge) in the graph. The goal is to capture the essence of the graph's topology, node connectivity, and other relevant features in a way that facilitates easier processing by machine learning algorithms, which often perform better with input data in a continuous vector space.</p><p dir="ltr">The main premise of kinematics-based force-directed graph embedding is that the nodes are considered as massive mobile objects that can be moved around in the embedding space under force. In this PhD thesis, we devised a general theoretical framework for the proposed graph embedding paradigm and provided the mathematical proof of convergence given the required constraints. From this point on, the objective was to explore force functions and parameters and methods of applying them in terms of their efficacy regarding graph embedding applications. We found some force functions that outperformed the state-of-the-art methods.</p><p dir="ltr">The author of this manuscript believes that the proposed paradigm will open a new chapter, specifically in the field of graph embedding and generally in the field of embedding.</p>

<p dir="ltr">With the increasing availability of inexpensive computing power in wireless radio network nodes, machine learning based models are being deployed in operations that traditionally relied on rule-based or statistical methods. Contemporary high bandwidth networks enable easy availability of significant amounts of training data in a comparatively short time, aiding in the development of better deep learning models. Specialized deep learning models developed for wireless networks have been shown to consistently outperform traditional methods in a variety of wireless network applications.</p><p><br></p><p dir="ltr">We aim to address some of the unique challenges inherent in the wireless radio communication domain. Firstly, as data is transmitted over the air, data privacy and adversarial attacks pose heightened risks. Secondly, due to the volume of data and the time-sensitive nature of the processing that is required, the speed of the machine learning model becomes a significant factor, often necessitating operation within a latency constraint. Thirdly, the impact of diverse and time-varying wireless environments means that any machine learning model also needs to be generalizable. The increasing computing power present in wireless nodes provides an opportunity to offload some of the deep learning to the edge, which also impacts data privacy.</p><p><br></p><p dir="ltr">Towards this goal, we work on deep learning methods that operate along different aspects of a wireless network—on network packets, error prediction, modulation classification, and channel estimation—and are able to operate within the latency constraint, while simultaneously providing better privacy and security. After proposing solutions that work in a traditional centralized learning environment, we explore edge learning paradigms where the learning happens in distributed nodes.</p>

<p dir="ltr">In recent years, advancements in robotics and computing power have enabled robots to master complex tasks. Nevertheless, merely executing tasks isn't sufficient for robots. To achieve higher robot autonomy, learning the objective function is crucial. Autonomous systems can effectively eliminate the need for explicit programming by autonomously learning the control objective and deriving their control policy through the observation of task demonstrations. Hence, there's a need to develop a method for robots to learn the desired objective functions. In this thesis, we address several challenges in objective learning for autonomous systems, enhancing the applicability of our method in real-world scenarios. The ultimate objective of the thesis is to create a universal objective learning approach capable of addressing a range of existing challenges in the field while emphasizing data efficiency and robustness. Hence, building upon the previously mentioned intuition, we present a framework for autonomous systems to address a variety of objective learning tasks in real-time, even in the presence of noisy data. In addition to objective learning, this framework is capable of handling various other learning and control tasks.</p><p dir="ltr">The first part of this thesis concentrates on objective learning methods, specifically inverse optimal control (IOC). Within this domain, we have made three significant contributions aimed at addressing three existing challenges in IOC: 1) learning from minimal data, 2) learning without prior knowledge of system dynamics, and 3) learning with system outputs. </p><p dir="ltr">The second part of this thesis aims to develop a unified IOC framework to address all the challenges previously mentioned. It introduces a new paradigm for autonomous systems, referred to as Online Control-Informed Learning. This paradigm aims to tackle various of learning and control tasks online with data efficiency and robustness to noisy data. Integrating optimal control theory, online state estimation techniques, and machine learning methods, our proposed paradigm offers an online learning framework capable of tackling a diverse array of learning and control tasks. These include online imitation learning, online system identification, and policy tuning on-the-fly, all with efficient use of data and computation resources while ensuring robust performance.</p>

<p dir="ltr">Advancements in machine learning, especially deep learning, have facilitated the creation of models capable of performing tasks previously thought impossible. This progress has opened new possibilities across diverse fields such as medical imaging and remote sensing. However, the performance of these models relies heavily on the availability of extensive labeled datasets.<br>Collecting large amounts of labeled data poses a significant financial burden, particularly in specialized fields like medical imaging and remote sensing, where annotation requires expert knowledge. To address this challenge, various methods have been developed to mitigate the necessity for labeled data or leverage information contained in unlabeled data. These encompass include self-supervised learning, few-shot learning, and semi-supervised learning. This dissertation centers on the application of semi-supervised learning in segmentation tasks.<br><br>We focus on panoptic segmentation, a task that combines semantic segmentation (assigning a class to each pixel) and instance segmentation (grouping pixels into different object instances). We choose two segmentation tasks in different domains: nerve segmentation in microscopic imaging and hyperspectral segmentation in satellite images from Mars.<br>Our study reveals that, while direct application of methods developed for natural images may yield low performance, targeted modifications or the development of robust models can provide satisfactory results, thereby unlocking new applications like machine-assisted annotation of new data.<br><br>This dissertation begins with a challenging panoptic segmentation problem in microscopic imaging, systematically exploring model architectures to improve generalization. Subsequently, it investigates how semi-supervised learning may mitigate the need for annotated data. It then moves to hyperspectral imaging, introducing a Hierarchical Bayesian model (HBM) to robustly classify single pixels. Key contributions of include developing a state-of-the-art U-Net model for nerve segmentation, improving the model's ability to segment different cellular structures, evaluating semi-supervised learning methods in the same setting, and proposing HBM for hyperspectral segmentation. <br>The dissertation also provides a dataset of labeled CRISM pixels and mineral detections, and a software toolbox implementing the full HBM pipeline, to facilitate the development of new models.</p>

<p dir="ltr">Scientific discovery of new knowledge from data can enhance our understanding of the physical world and lead to the innovation of new technologies. AI-driven methods can greatly accelerate scientific discovery and are essential for analyzing and identifying patterns in huge volumes of experimental data. However, current AI-driven scientific discovery pipeline suffers from several inefficiencies including but not limited to lack of <b>precise modeling</b>, lack of <b>efficient learning methods</b>, and lack of <b>human-in-the-loop integrated frameworks</b> in the scientific discovery loop. Such inefficiencies increase resource requirements such as expensive computing infrastructures, significant human expert efforts and subsequently slows down scientific discovery.</p><p dir="ltr">In this thesis, I introduce a collection of methods to address the lack of precise modeling, lack of efficient learning methods and lack of human-in-the-loop integrated frameworks in AI-driven scientific discovery workflow. These methods include automatic physics model learning from partially annotated noisy video data, accelerated partial differential equation (PDE) physics model learning, and an integrated AI-driven platform for rapid analysis of experimental video data. <b>My research has led to the discovery of a new size fluctuation property of material defects</b> exposed to high temperature and high irradiation environments such as inside nuclear reactors. Such discovery is essential for designing strong materials that are critical for energy applications.</p><p dir="ltr">To address the lack of precise modeling of physics learning tasks, I developed NeuraDiff, an end-to-end method for learning phase field physics models from noisy video data. In previous learning approaches involving multiple disjoint steps, errors in one step can propagate to another, thus affecting the accuracy of the learned physics models. Trial-and-error simulation methods for learning physics model parameters are inefficient, heavily dependent on expert intuition and may not yield reasonably accurate physics models even after many trial iterations. By encoding the physics model equations directly into learning, end-to-end NeuraDiff framework can provide <b>~100%</b> accurate tracking of material defects and yield correct physics model parameters. </p><p dir="ltr">To address the lack of efficient methods for PDE physics model learning, I developed Rapid-PDE and Reel. The key idea behind these methods is the random projection based compression of system change signals which are sparse in - either value domain (Rapid-PDE) or, both value and frequency domain (Reel). Experiments show that PDE model training times can be reduced significantly using our Rapid-PDE (<b>50-70%)</b> and Reel (<b>70-98%</b>) methods. </p><p dir="ltr">To address the lack of human-in-the-loop integrated frameworks for high volume experimental data analysis, I developed an integrated framework with an easy-to-use annotation tool. Our interactive AI-driven annotation tool can reduce video annotation times by <b>50-75%</b>, and enables material scientists to scale up the analysis of experimental videos.</p><p dir="ltr"><b>Our framework for analyzing experimental data has been deployed in the real world</b> for scaling up in-situ irradiation experiment video analysis and has played a crucial role in the discovery of size fluctuation of material defects under extreme heat and irradiation. </p>

<p>A wide range of optoelectronic applications demand semiconductors optimized for purpose.</p> <p>My research focused on data-driven identification of ABX3 Halide perovskite compositions for optimum photovoltaic absorption in solar cells.</p> <p>I trained machine learning models on previously reported datasets of halide perovskite band gaps based on first principles computations performed at different fidelities.</p> <p>Using these, I identified mixtures of candidate constituents at the A, B or X sites of the perovskite supercell which leveraged how mixed perovskite band gaps deviate from the linear interpolations predicted by Vegard's law of mixing to obtain a selection of stable perovskites with band gaps in the ideal range of 1 to 2 eV for visible light spectrum absorption.</p> <p>These models predict the perovskite band gap using the composition and inherent elemental properties as descriptors.</p> <p>This enables accurate, high fidelity prediction and screening of the much larger chemical space from which the data samples were drawn.</p> <p><br></p> <p>I utilized a recently published density functional theory (DFT) dataset of more than 1300 perovskite band gaps from four different levels of theory, added to an experimental perovskite band gap dataset of \textasciitilde{}100 points, to train random forest regression (RFR), Gaussian process regression (GPR), and Sure Independence Screening and Sparsifying Operator (SISSO) regression models, with data fidelity added as one-hot encoded features.</p> <p>I found that RFR yields the best model with a band gap root mean square error of 0.12 eV on the total dataset and 0.15 eV on the experimental points.</p> <p>SISSO provided compound features and functions for direct prediction of band gap, but errors were larger than from RFR and GPR.</p> <p>Additional insights gained from Pearson correlation and Shapley additive explanation (SHAP) analysis of learned descriptors suggest the RFR models performed best because of (a) their focus on identifying and capturing relevant feature interactions and (b) their flexibility to represent nonlinear relationships between such interactions and the band gap.</p> <p>The best model was deployed for predicting experimental band gap of 37785 hypothetical compounds.</p> <p>Based on this, we identified 1251 stable compounds with band gap predicted to be between 1 and 2 eV at experimental accuracy, successfully narrowing the candidates to about 3% of the screened compositions.</p>

<p>Our modern society is more dependent on its assets in space now more than ever. For<br> example, the Global Positioning System (GPS) many rely on for navigation uses data from a<br> 24-satellite constellation. Additionally, our current infrastructure for gas pumps, cell phones,<br> ATMs, traffic lights, weather data, etc. all depend on satellite data from various constel-<br> lations. As a result, it is increasingly necessary to accurately track and predict the space<br> domain. In this thesis, after discussing how space object tracking and object position pre-<br> diction is currently being done, I propose a machine learning-based approach to improving<br> the space object position prediction over the standard SGP4 method, which is limited in<br> prediction accuracy time to about 24 hours. Using this approach, we are able to show that<br> meaningful improvements over the standard SGP4 model can be achieved using a machine<br> learning model built based on a type of recurrent neural network called a long short term<br> memory model (LSTM). I also provide distance predictions for 4 different space objects over<br> time frames of 15 and 30 days. Future work in this area is likely to include extending and<br> validating this approach on additional satellites to construct a more general model, testing a<br> wider range of models to determine limits on accuracy across a broad range of time horizons,<br> and proposing similar methods less dependent on antiquated data formats like the TLE.</p>

<p>The most important networks from different domains—such as Computing, Organization, Economic, Social, Academic, and Biology—are networks that change over time. For example, in an organization there are email and collaboration networks (e.g., different people or teams working on a document). Apart from the connectivity of the networks changing over time, they can contain attributes such as the topic of an email or message, contents of a document, or the interests of a person in an academic citation or a social network. Analyzing these dynamic networks can be critical in decision-making processes. For instance, in an organization, getting insight into how people from different teams collaborate, provides important information that can be used to optimize workflows.</p> <p><br></p> <p>Network generative models provide a way to study and analyze networks. For example, benchmarking model performance and generalization in tasks like node classification, can be done by evaluating models on synthetic networks generated with varying structure and attribute correlation. In this work, we begin by presenting our systemic study of the impact that graph structure and attribute auto-correlation on the task of node classification using collective inference. This is the first time such an extensive study has been done. We take advantage of a recently developed method that samples attributed networks—although static—with varying network structure jointly with correlated attributes. We find that the graph connectivity that contributes to the network auto-correlation (i.e., the local relationships of nodes) and density have the highest impact on the performance of collective inference methods.</p> <p><br></p> <p>Most of the literature to date has focused on static representations of networks, partially due to the difficulty of finding readily-available datasets of dynamic networks. Dynamic network generative models can bridge this gap by generating synthetic graphs similar to observed real-world networks. Given that motifs have been established as building blocks for the structure of real-world networks, modeling them can help to generate the graph structure seen and capture correlations in node connections and activity. Therefore, we continue with a study of motif evolution in <em>dynamic</em> temporal graphs. Our key insight is that motifs rarely change configurations in fast-changing dynamic networks (e.g. wedges intotriangles, and vice-versa), but rather keep reappearing at different times while keeping the same configuration. This finding motivates the generative process of our proposed models, using temporal motifs as building blocks, that generates dynamic graphs with links that appear and disappear over time.</p> <p><br></p> <p>Our first proposed model generates dynamic networks based on motif-activity and the roles that nodes play in a motif. For example, a wedge is sampled based on the likelihood of one node having the role of hub with the two other nodes being the spokes. Our model learns all parameters from observed data, with the goal of producing synthetic graphs with similar graph structure and node behavior. We find that using motifs and node roles helps our model generate the more complex structures and the temporal node behavior seen in real-world dynamic networks.</p> <p><br></p> <p>After observing that using motif node-roles helps to capture the changing local structure and behavior of nodes, we extend our work to also consider the attributes generated by nodes’ activities. We propose a second generative model for attributed dynamic networks that (i) captures network structure dynamics through temporal motifs, and (ii) extends the structural roles of nodes in motifs to roles that generate content embeddings. Our new proposed model is the first to generate synthetic dynamic networks and sample content embeddings based on motif node roles. To the best of our knowledge, it is the only attributed dynamic network model that can generate <em>new</em> content embeddings—not observed in the input graph, but still similar to that of the input graph. Our results show that modeling the network attributes with higher-order structures (e.g., motifs) improves the quality of the networks generated.</p> <p><br></p> <p>The generative models proposed address the difficulty of finding readily-available datasets of dynamic networks—attributed or not. This work will also allow others to: (i) generate networks that they can share without divulging individual’s private data, (ii) benchmark model performance, and (iii) explore model generalization on a broader range of conditions, among other uses. Finally, the evaluation measures proposed will elucidate models, allowing fellow researchers to push forward in these domains.</p>