We show how physics-informed neural networks can be used to solve compressible \NS equations in protoplanetary disks. While young planets form in protoplanetary disks, because of the limitation of current techniques, direct observations of them are challenging. So instead, existing methods infer the presence and properties of planets from the disk structures created by disk-planet interactions. Hydrodynamic and radiative transfer simulations play essential roles in this process. Currently, the lack of computer resources for these expensive simulations has become one of the field's main bottlenecks. To solve this problem, we explore the possibility of using physics-informed neural networks, a machine learning method that trains neural networks using physical laws, to substitute the simulations. We identify three main bottlenecks that prevent the physics-informed neural networks from achieving this goal, which we overcome by hard-constraining initial conditions, scaling outputs and balancing gradients. With these improvements, we reduce the relative L2 errors of predicted solutions by 97% ~ 99\% compared to the vanilla PINNs on solving compressible NS equations in protoplanetary disks. / Graduate / 2022-12-10

While the second part of the 20th century was dominated by combustion engine powered vehicles, climate change and limited oil resources has been forcing car manufacturers and other companies in the mobility sector to switch to renewable energy sources. Electric engines supplied by Li-ion battery cells are on the forefront of this revolution in the mobility sector. A challenging but very important task hereby is the precise forecasting of the degradation of battery state-of-health and state-of-charge. Hence, there is a high demand in models that can predict the SOH and SOC and consider the specifics of a certain kind of battery cell and the usage profile of the battery. While traditional physics-based and data-driven approaches are used to monitor the SOH and SOC, they both have limitations related to computational costs or that require engineers to continually update their prediction models as new battery cells are developed and put into use in battery-powered vehicle fleets. In this dissertation, we enhance a hybrid physics-informed machine learning version of a battery SOC model to predict voltage drop during discharge. The enhanced model captures the effect of wide variation of load levels, in the form of input current, which causes large thermal stress cycles. The cell temperature build-up during a discharge cycle is used to identify temperature-sensitive model parameters. Additionally, we enhance an aging model built upon cumulative energy drawn by introducing the effect of the load level. We then map cumulative energy and load level to battery capacity with a Gaussian process model. To validate our approach, we use a battery aging dataset collected on a self-developed testbed, where we used a wide current level range to age battery packs in accelerated fashion. Prediction results show that our model can be successfully calibrated and generalizes across all applied load levels.

This thesis is divided into two parts dealing with the optimization problems in Markov decision processes (MDPs) and different neural network-based numerical solvers for partial differential equations (PDEs). In Part I we analyze the optimization problem arising in (partially observable) Markov decision processes using tools from algebraic statistics and information geometry, which can be viewed as neighboring fields of applied algebra and differential geometry, respectively. Here, we focus on infinite horizon problems and memoryless stochastic policies. Markov decision processes provide a mathematical framework for sequential decision-making on which most current reinforcement learning algorithms are built. They formalize the task of optimally controlling the state of a system through appropriate actions. For fully observable problems, the action can be selected knowing the current state of the system. This case has been studied extensively and optimizing the action selection is known to be equivalent to solving a linear program over the (generalized) stationary distributions of the Markov decision process, which are also referred to as state-action frequencies. In Chapter 3, we study partially observable problems where an action must be chosen based solely on an observation of the current state, which might not fully reveal the underlying state. We characterize the feasible state-action frequencies of partially observable Markov decision processes by polynomial inequalities. In particular, the optimization problem in partially observable MDPs is described as a polynomially constrained linear objective program that generalizes the (dual) linear programming formulation of fully observable problems. We use this to study the combinatorial and algebraic complexity of this optimization problem and to upper bound the number of critical points over the individual boundary components of the feasible set. Furthermore, we show that our polynomial programming formulation can be used to effectively solve partially observable MDPs using interior point methods, numerical algebraic techniques, and convex relaxations. Gradient-based methods, including variants of natural gradient methods, have gained tremendous attention in the theoretical reinforcement learning community, where they are commonly referred to as (natural) policy gradient methods. In Chapter 4, we provide a unified treatment of a variety of natural policy gradient methods for fully observable problems by studying their state-action frequencies from the standpoint of information geometry. For a variety of NPGs and reward functions, we show that the trajectories in state-action space are solutions of gradient flows with respect to Hessian geometries, based on which we obtain global convergence guarantees and convergence rates. In particular, we show linear convergence for unregularized and regularized NPG flows with the metrics proposed by Morimura and co-authors and Kakade by observing that these arise from the Hessian geometries of the entropy and conditional entropy, respectively. Further, we obtain sublinear convergence rates for Hessian geometries arising from other convex functions like log-barriers. We provide experimental evidence indicating that our predicted rates are essentially tight. Finally, we interpret the discrete-time NPG methods with regularized rewards as inexact Newton methods if the NPG is defined with respect to the Hessian geometry of the regularizer. This yields local quadratic convergence rates of these methods for step size equal to the inverse penalization strength, which recovers existing results as special cases. Part II addresses neural network-based PDE solvers that have recently experienced tremendous growth in popularity and attention in the scientific machine learning community. We focus on two approaches that represent the approximation of a solution of a PDE as the minimization over the parameters of a neural network: the deep Ritz method and physically informed neural networks. In Chapter 5, we study the theoretical properties of the boundary penalty for these methods and obtain a uniform convergence result for the deep Ritz method for a large class of potentially nonlinear problems. For linear PDEs, we estimate the error of the deep Ritz method in terms of the optimization error, the approximation capabilities of the neural network, and the strength of the penalty. This reveals a trade-off in the choice of the penalization strength, where too little penalization allows large boundary values, and too strong penalization leads to a poor solution of the PDE inside the domain. For physics-informed networks, we show that when working with neural networks that have zero boundary values also the second derivatives of the solution are approximated whereas otherwise only lower-order derivatives are approximated. In Chapter 6, we propose energy natural gradient descent, a natural gradient method with respect to second-order information in the function space, as an optimization algorithm for physics-informed neural networks and the deep Ritz method. We show that this method, which can be interpreted as a generalized Gauss-Newton method, mimics Newton’s method in function space except for an orthogonal projection onto the tangent space of the model. We show that for a variety of PDEs, natural energy gradients converge rapidly and approximations to the solution of the PDE are several orders of magnitude more accurate than gradient descent, Adam and Newton’s methods, even when these methods are given more computational time.:Chapter 1. Introduction 1 1.1 Notation and conventions 7 Part I. Geometry of Markov decision processes 11 Chapter 2. Background on Markov decision processes 12 2.1 State-action frequencies 19 2.2 The advantage function and Bellman optimality 23 2.3 Rational structure of the reward and an explicit line theorem 26 2.4 Solution methods for Markov decision processes 35 Chapter 3. State-action geometry of partially observable MDPs 44 3.1 The state-action polytope of fully observables systems 45 3.2 State-action geometry of partially observable systems 54 3.3 Number and location of critical points 69 3.4 Reward optimization in state-action space (ROSA) 83 Chapter 4. Geometry and convergence of natural policy gradient methods 94 4.1 Natural gradients 96 4.2 Natural policy gradient methods 101 4.3 Convergence of natural policy gradient flows 107 4.4 Locally quadratic convergence for regularized problems 128 4.5 Discussion and outlook 131 Part II. Neural network-based PDE solvers 133 Chapter 5. Theoretical analysis of the boundary penalty method for neural network-based PDE solvers 134 5.1 Presentation and discussion of the main results 137 5.2 Preliminaries regarding Sobolev spaces and neural networks 146 5.3 Proofs regarding uniform convergence for the deep Ritz method 150 5.4 Proofs of error estimates for the deep Ritz method 156 5.5 Proofs of implications of exact boundary values in residual minimization 167 Chapter 6. Energy natural gradients for neural network-based PDE solvers 174 6.1 Energy natural gradients 176 6.2 Experiments 183 6.3 Conclusion and outlook 192 Bibliography 193

[pt] Métodos de aprendizado de máquina que incorporam o conhecimentoda física do problema em um aprendizado baseado em dados têm se tornadopromissores para a representação e previsão de sistemas não-lineares de escoamento de fluidos com múltiplas escalas no espaço e no tempo. Este trabalhoaborda um desses métodos, a Inferência de Operadores (OpInf), no contextode redução da ordem de modelos. Ao resolver um problema de regressão multivariável em espaço latente, cuja base é calculada através de uma decomposiçãoortogonal apropriada (POD) do respectivo conjunto de dados de alta fidelidade,o OpInf procura por operadores ótimos de baixa dimensão que representama dinâmica do sistema. Entretanto, o método ainda requer melhorias em suaestratégia de regularização e em sua confiabilidade para cenários complexos,assim como na robustez dos modelos treinados a partir de dados limitados.Para isso, um algoritmo eficiente e recente para a busca de hiperparâmetros,uma inferência de operadores sequencial e uma estratégia de aprendizagem emconjunto foram implementados com sucesso no presente trabalho. Outras modificações do OpInf padrão também foram investigadas, tais como uma reduçãode dados incremental e termos forçantes baseados em POD. Para testá-las,diferentes sistemas físicos foram considerados: condução de calor transiente;escoamento em cavidade com parede oscilante; propagação de onda não-linear;convecção natural; dispersão de CO2 atmosférico e elevação da altura da superfície do mar por maré de tempestade. De modo geral, foi demonstrado queos modelos baseados no OpInf podem ter capacidades preditivas muito boaspara escoamentos altamente turbulentos e também para sistemas físicos paramétricos. Além disso, mostrou-se que tais modelos podem ser empregadosem previsões climáticas rápidas uma vez que eles são capazes de lidar comdados geoespaciais ruidosos. Finalmente, os resultados sugerem que o OpInfpode ser uma alternativa confiável às redes neurais de aprendizado profundopara redução da ordem de modelos em decorrência de seus menores custoscomputacionais e bom desempenho para além do horizonte de treinamento. / [en] Scientific machine learning methods that incorporate physics knowledge on a data-driven learning have become quite promising for the representation and prediction of nonlinear fluid flow systems with multiple scales in space and time. This work addresses one of these methods, the Operator Inference (OpInf), in the context of model order reduction. By solving a multivariable regression problem in latent space, whose basis is computed through a proper orthogonal decomposition (POD) of the respective high-fidelity dataset, the OpInf seeks for optimal low-dimensional operators that represent the system dynamics. However, this method still requires improvements in its regularization strategy and reliability in complex scenarios, as well as in the robustness of the obtained reduced models for long-term extrapolation when trained with limited data. For that, a recent and efficient algorithm for hyperparameters search, a sequential operator inference and an ensemble learning strategy were successfully implemented in the present work. Other modifications to the standard OpInf were also investigated, such as an incremental data reduction and POD-based forcing terms. To test them, different physical systems were considered: transient heat conduction; oscillating lid-driven cavity flow; nonlinear wave propagation; natural convection; atmospheric CO2 dispersion and sea surface height elevation due to tidal surges. Overall, it was demonstrated OpInf-based models may have very good predictive capabilities for highly turbulent flows and parameter-dependent systems. Furthermore, it was shown these models may be employed for fast response climate-related predictions as they are capable to handle noisy geospatial measurements. Finally, the results suggest the OpInf may be a reliable alternative to deep learning neural networks for model order reduction due to its lower computational costs and good performance beyond the training horizon.

<p dir="ltr">The integration of scientific machine learning with computational structural mechanics offers a range of opportunities to address some of the most significant challenges currently experienced by multiphysical simulations, design optimization, and inverse sensing problems. While traditional mesh-based numerical methods, such as the Finite Element Method (FEM), have proven to be very powerful when applied to complex and geometrically inhomogeneous domains, their performance deteriorates very rapidly when faced with simulation scenarios involving high-dimensional systems, high-frequency inputs and outputs, and highly irregular domains. All these elements contribute to increase in the overall computational cost, the mesh dependence, and the number of costly matrix operations that can rapidly render FEM inapplicable. In a similar way, traditional inverse solvers, including global optimization methods, also face important limitations when handling high-dimensional, dynamic design spaces, and multiphysics systems. Recent advances in machine learning (ML) and deep learning have opened new ways to develop alternative techniques for the simulation of complex engineering systems. However, most of the existing deep learning methods are data greedy, a property that strides with the typically limited availability of physical observations and data in scientific applications. This sharp contrast between needed and available data can lead to poor approximations and physically inconsistent solutions. An opportunity to overcome this problem is offered by the class of so-called physics-informed or scientific machine learning methods that leverage the knowledge of problem-specific governing physics to alleviate, or even completely eliminate, the dependence on data. As a result, this class of methods can leverage the advantages of ML algorithms without inheriting their data greediness. This dissertation aims to develop scientific ML methods for application to forward and inverse problems in acoustics and structural mechanics while simultaneously overcoming some of the most significant limitations of traditional computational mechanics methods. </p><p dir="ltr">This work develops fully physics-driven deep learning frameworks specifically conceived to perform forward <i>simulations</i> of mechanical systems that provide approximate, yet physically consistent, solutions without requiring labeled data. The proposed set of approaches is characterized by low discretization dependence and is conceived to support parallel computations in future developments. These characteristics make these methods efficient to handle high degrees of freedom systems, high-frequency simulations, and systems with irregular geometries. The proposed deep learning frameworks enforce the governing equations within the deep learning algorithm, therefore removing the need for costly training data generation while preserving the physical accuracy of the simulation results. Another noteworthy contribution consists in the development of a fully physics-driven deep learning framework capable of improving the computational time for simulating domains with irregular geometries by orders of magnitude in comparison to the traditional mesh-based methods. This novel framework is both geometry-aware and maintains physical consistency throughout the simulation process. The proposed framework displays the remarkable ability to simulate systems with different domain geometries without the need for a new model assembly or a training phase. This capability is in stark contrast with current numerical mesh-based methods, that require new model assembly, and with conventional ML models, that require new training.</p><p dir="ltr">In the second part of this dissertation, the work focuses on the development of ML-based approaches to solve inverse problems. A new deep reinforcement learning framework tailored for dynamic <i>design optimization</i> tasks in coupled-physics problems is presented. The framework effectively addresses key limitations of traditional methods by enabling the exploration of high-dimensional design spaces and supporting sequential decision-making in complex multiphysics systems. Maintaining the focus on the class of inverse problems, ML-based algorithms for <i>remote sensing</i> are also explored with particular reference to structural health monitoring applications. A modular neural network framework is formulated by integrating three essential modules: physics-based regularization, geometry-based regularization, and reduced-order representation. The concurrent use of these modules has shown remarkable performance when addressing the challenges associated with nonlinear, high-dimensional, and often ill-posed remote sensing problems. Finally, this dissertation illustrates the efficacy of deep learning approaches for experimental remote sensing. Results show the significant ability of these techniques when applied to learning inverse mappings based on high-dimensional and noisy experimental data. The proposed framework incorporates data augmentation and denoising techniques to handle limited and noisy experimental datasets, hence establishing a robust approach for training on experimental data.</p>