<p dir="ltr">Despite their astonishing capacity to fit data, neural networks have difficulties extrapolating beyond training data distribution. When the out-of-distribution prediction task is formalized as a counterfactual query on a causal model, the reason for their extrapolation failure is clear: neural networks learn spurious correlations in the training data rather than features that are causally related to the target label. This thesis proposes to perform a causal search over a known family of causal models to learn robust (maximally invariant) predictors for single- and multiple-environment extrapolation tasks.</p><p dir="ltr">First, I formalize the out-of-distribution task as a counterfactual query over a structural causal model. For single-environment extrapolation, I argue that symmetries of the input data are valuable for training neural networks that can extrapolate. I introduce Asymmetry learning, a new learning paradigm that is guided by the hypothesis that all (known) symmetries are mandatory even without evidence in training, unless the learner deems it inconsistent with the training data. Asymmetry learning performs a causal model search to find the simplest causal model defining a causal connection between the target labels and the symmetry transformations that affect the label. My experiments on a variety of out-of-distribution tasks on images and sequences show that proposed methods extrapolate much better than the standard neural networks.</p><p dir="ltr">Then, I consider multiple-environment out-of-distribution tasks in dynamical system forecasting that arise due to shifts in initial conditions or parameters of the dynamical system. I identify key OOD challenges in the existing deep learning and physics-informed machine learning (PIML) methods for these tasks. To mitigate these drawbacks, I combine meta-learning and causal structure discovery over a family of given structural causal models to learn the underlying dynamical system. In three simulated forecasting tasks, I show that the proposed approach is 2x to 28x more robust than the baselines.</p>

Physics Informed Machine Learning (PIML) has emerged as the forefront of research in scientific machine learning with the key motivation of systematically coupling machine learning (ML) methods with prior domain knowledge often available in the form of physics supervision. Uncertainty quantification (UQ) is an important goal in many scientific use-cases, where the obtaining reliable ML model predictions and accessing the potential risks associated with them is crucial. In this thesis, we propose novel methodologies in three key areas for improving uncertainty quantification for PIML. First, we propose to explicitly infuse the physics prior in the form of monotonicity constraints through architectural modifications in neural networks for quantifying uncertainty. Second, we demonstrate a more general framework for quantifying uncertainty with PIML that is compatible with generic forms of physics supervision such as PDEs and closed form equations. Lastly, we study the limitations of physics-based loss in the context of Physics-informed Neural Networks (PINNs), and develop an efficient sampling strategy to mitigate the failure modes. / Doctor of Philosophy / Owing to the success of deep learning in computer vision and natural language processing there is a growing interest of using deep learning in scientific applications. In scientific applications, knowledge is available in the form of closed form equations, partial differential equations, etc. along with labeled data. My work focuses on developing deep learning methods that integrate these forms of supervision. Especially, my work focuses on building methods that can quantify uncertainty in deep learning models, which is an important goal for high-stakes applications.

This study presents a procedure for the estimation of stopping behavior of industrial robots with a trained neural network. This trained network is presented as a single channel in a redundant architecture for safety control applications, where its potential for future integration with an analytical model of robot stopping is discussed. Basic physical relations for simplified articulated manipulators are derived, which motivate a choice of quantities to predict robot stopping behavior and inform the training and testing of a network for prediction of stopping distances and times. Robot stopping behavior is considered in the context of relevant standards ISO 10218-1, ISO/TS 15066 and IS0 13849-1, which inform the definitions for safety related stopping distances and times used in this study. Prior work on the estimation of robot stopping behavior is discussed alongside applications of machine learning to the broader field of industrial robotics, and particularly to the cases of prediction of forward and inverse kinematics with trained networks. A state-driven data collection program is developed to perform repeated stopping experiments for a controlled stop on path within a specified sampling domain. This program is used to collect data for a simulated and real robot system. Special attention is given to the identification of meaningful stopping times, which includes the separation of stopping into pre-deceleration and post-deceleration phases. A definition is provided for stopping of a robot in a safety context, based on the observation that residual motion over short distances (less than 1 mm) and at very low velocities (less than 1 mm/s) is not relevant to robot safety. A network architecture and hyperparameters are developed for the prediction of stopping distances and times for the first three joints of the manipulator without the inclusion of payloads. The result is a dual-network structure, where stopping distance predictions from the distance prediction network serve as inputs to the stopping time prediction network. The networks are validated on their capacity to interpolate and extrapolate predictions of robot stopping behavior in the presence of initial conditions not included in the training and testing data. A method is devised for the calculation of prediction errors for training training, testing and validation data. This method is applied both to interpolation and extrapolation to new initial velocity and positional conditions of the manipulator. In prediction of stopping distances and times, the network is highly successful at interpolation, resulting in comparable or nominally higher errors for the validation data set when compared to the errors for training and testing data. In extrapolation to new initial velocity and positional conditions, notably higher errors in the validation data predictions are observed for the networks considered. Future work in the areas of predictions of stopping behavior with payloads and tooling, further applications to collaborative robotics, analytical models of stopping behavior, inclusion of additional stopping functions, use of explainable AI methods and physics-informed networks are discussed. / Master of Science / As the uses for industrial robots continue to grow and expand, so do the need for robust safety measures to avoid, control, or limit the risks posed to human operators and collaborators. This is exemplified by Isaac Asimov's famous first law of robotics - "A robot may not injure a human being, or, through inaction, allow a human being to come to harm." As applications for industrial robots continue to expand, it is beneficial for robots and human operators to collaborate in work environments without fences. In order to ethically implement such increasingly complex and collaborative industrial robotic systems, the ability to limit robot motion with safety functions in a predictable and reliable way (as outlined by international standards) is paramount. In the event of either a technical failure (due to malfunction of sensors or mechanical hardware) or change in environmental conditions, it is important to be able to stop an industrial robot from any position in a safe and controlled manner. This requires real-time knowledge of the stopping distance and time for the manipulator. To understand stopping distances and times reliability, multiple independent methods can be used and compared to predict stopping behavior. The use of machine learning methods is of particular interest in this context due to their speed of processing and the potential for basis on real recorded data. In this study, we will attempt to evaluate the efficacy of machine learning algorithms to predict stopping behavior and assess their potential for implementation alongside analytical models. A reliable, multi-method approach for estimating stopping distances and times could also enable further methods for safety in collaborative robotics such as Speed and Separation Monitoring (SSM), which monitors both human and robot positions to ensure that a safe stop is always possible. A program for testing and recording the stopping distances and times for the robot is developed. As stopping behavior varies based on the positions and speeds of the robot at the time of stopping, a variety of these criteria are tested with the robot stopping program. This data is then used to train an artificial neural network, a machine learning method that mimics the structure of human and animal brains to learn relationships between data inputs and outputs. This network is used to predict both the stopping distance and time of the robot. The network is shown to produce reasonable predictions, especially for positions and speeds that are intermediate to those used to train the network. Future improvements are suggested and a method is suggested for use of stopping distance and time quantities in robot safety applications.

<p dir="ltr">Evidence for climate change-related impacts and risks is already widespread globally, affecting not only the ecosystems but also the economy and health of our communities. Data-driven predictive modeling approaches such as machine learning and deep learning have emerged to be powerful tools for interpreting large and complex non-linear datasets such as meteorological variables from weather stations or the distribution of infectious droplets produced in a cough. However, the strength of these data-driven models can be further optimized by complementing them with foundational knowledge of the physical processes they represent. By understanding the core physics, one can enhance the reliability and accuracy of predictive outcomes. The effectiveness of these combined approaches becomes particularly feasible and robust with the recent advancements in the High-Performance Computing field. With improved processing speed, algorithm design, and storage capabilities, modern computers allow for a deeper and more precise examination of the data. Such advancements equip us to address the diverse challenges presented by climate change more effectively.</p><p dir="ltr">In particular, this document advances research in mitigating and preventing the consequences of global warming by implementing data-driven predictive models based on statistical, machine learning, and deep learning methods via two phases. In the first phase, this dissertation proposes frameworks consisting of machine and deep learning algorithms to increase the resilience of small-scale renewable energy systems, which are essential for reducing greenhouse gas emissions in the ecosystems. The second phase focuses on using data from physics-based models, i.e., computational fluid dynamics (CFD), in data-driven predictive models for improving the design of air cleaning technologies, which are crucial to reducing the transmission of infectious diseases in indoor environments. </p><p dir="ltr">Specifically, this work is an article-based collection of published (or will be published) research articles. The articles are reformatted to fit the thesis's structure. The contents of the original articles are self-contained. </p>