Subsurface porous media, like aquifers, petroleum reservoirs, and geothermal systems, are vital for natural resources and environmental management. Extensive research has been conducted to understand flow and transport in these media, addressing challenges in hydrocarbon extraction, carbon storage and waste management. Classifying the type of porous media (e.g., sandstone, carbonate) is often the first step in the rock characterization process, and it provides critical information regarding the physical properties of the porous media. Therefore, we utilize multivariate statistical methods with discriminant analysis to categorize porous media samples which proved to be efficient by achieving excellent classification accuracy on testing datasets and served as a surrogate tool to study key porous media characteristics. While recent advances in three-dimensional (3D) imaging of core samples have enabled digital subsurface characterization, the exorbitant computational cost associated with direct numerical simulation in 3D remains a persistent challenge. In contrast, machine learning (ML) models are much more efficient, though their use in subsurface characterization is still in its infancy. Therefore, we introduce a novel 3D convolution neural network (CNN) for end-to-end prediction of permeability. By increasing dataset size, diversity, and optimizing the network architecture, our model surpasses the accuracy of existing 3D CNN models for permeability prediction. It demonstrates excellent generalizability, accurately predicting permeability in previously unseen samples. However, despite the efficiency of the developed 3D CNN model for accurate and fast permeability prediction, its utility remains limited to small subdomains of the digital rock samples. Therefore, we introduce an upscaling technique using a new analytical solution to calculate effective permeability in a 3D digital rock composed of 2 × 2 × 2 anisotropic cells. By incorporating this solution into physics-informed neural network (PINN) models, we achieve highly accurate results. Even when upscaling previously unseen samples at multiple levels, the PINN with the physics-informed module maintains excellent accuracy. This advancement enhances the capability of ML models, like 3D CNN, for efficient and accurate digital rock analysis at the core scale. After successfully applying ML models in permeability prediction, we now extend their application to another important parameter in subsurface engineering projects: effective thermal conductivity, which is a key parameter in engineering projects like radioactive waste repositories, geothermal energy production, and underground energy storage. To address the need for large training data and processing power in ML models, we propose a novel framework based on transfer learning. This approach allows prior knowledge from previous applications to be transferred, resulting in faster and more efficient implementation of new relevant applications. We introduce CNN models trained on various porous media samples that leverage transfer learning to predict porous media sample thermal conductivity accurately. Our approach reduces training time, processing power, and data requirements, enabling effective prediction and analysis of porous media properties such as permeability and thermal conductivity. It also facilitates the application of ML to other properties, improving efficiency and accuracy. / Thesis / Doctor of Philosophy (PhD)

This thesis is divided into two parts dealing with the optimization problems in Markov decision processes (MDPs) and different neural network-based numerical solvers for partial differential equations (PDEs). In Part I we analyze the optimization problem arising in (partially observable) Markov decision processes using tools from algebraic statistics and information geometry, which can be viewed as neighboring fields of applied algebra and differential geometry, respectively. Here, we focus on infinite horizon problems and memoryless stochastic policies. Markov decision processes provide a mathematical framework for sequential decision-making on which most current reinforcement learning algorithms are built. They formalize the task of optimally controlling the state of a system through appropriate actions. For fully observable problems, the action can be selected knowing the current state of the system. This case has been studied extensively and optimizing the action selection is known to be equivalent to solving a linear program over the (generalized) stationary distributions of the Markov decision process, which are also referred to as state-action frequencies. In Chapter 3, we study partially observable problems where an action must be chosen based solely on an observation of the current state, which might not fully reveal the underlying state. We characterize the feasible state-action frequencies of partially observable Markov decision processes by polynomial inequalities. In particular, the optimization problem in partially observable MDPs is described as a polynomially constrained linear objective program that generalizes the (dual) linear programming formulation of fully observable problems. We use this to study the combinatorial and algebraic complexity of this optimization problem and to upper bound the number of critical points over the individual boundary components of the feasible set. Furthermore, we show that our polynomial programming formulation can be used to effectively solve partially observable MDPs using interior point methods, numerical algebraic techniques, and convex relaxations. Gradient-based methods, including variants of natural gradient methods, have gained tremendous attention in the theoretical reinforcement learning community, where they are commonly referred to as (natural) policy gradient methods. In Chapter 4, we provide a unified treatment of a variety of natural policy gradient methods for fully observable problems by studying their state-action frequencies from the standpoint of information geometry. For a variety of NPGs and reward functions, we show that the trajectories in state-action space are solutions of gradient flows with respect to Hessian geometries, based on which we obtain global convergence guarantees and convergence rates. In particular, we show linear convergence for unregularized and regularized NPG flows with the metrics proposed by Morimura and co-authors and Kakade by observing that these arise from the Hessian geometries of the entropy and conditional entropy, respectively. Further, we obtain sublinear convergence rates for Hessian geometries arising from other convex functions like log-barriers. We provide experimental evidence indicating that our predicted rates are essentially tight. Finally, we interpret the discrete-time NPG methods with regularized rewards as inexact Newton methods if the NPG is defined with respect to the Hessian geometry of the regularizer. This yields local quadratic convergence rates of these methods for step size equal to the inverse penalization strength, which recovers existing results as special cases. Part II addresses neural network-based PDE solvers that have recently experienced tremendous growth in popularity and attention in the scientific machine learning community. We focus on two approaches that represent the approximation of a solution of a PDE as the minimization over the parameters of a neural network: the deep Ritz method and physically informed neural networks. In Chapter 5, we study the theoretical properties of the boundary penalty for these methods and obtain a uniform convergence result for the deep Ritz method for a large class of potentially nonlinear problems. For linear PDEs, we estimate the error of the deep Ritz method in terms of the optimization error, the approximation capabilities of the neural network, and the strength of the penalty. This reveals a trade-off in the choice of the penalization strength, where too little penalization allows large boundary values, and too strong penalization leads to a poor solution of the PDE inside the domain. For physics-informed networks, we show that when working with neural networks that have zero boundary values also the second derivatives of the solution are approximated whereas otherwise only lower-order derivatives are approximated. In Chapter 6, we propose energy natural gradient descent, a natural gradient method with respect to second-order information in the function space, as an optimization algorithm for physics-informed neural networks and the deep Ritz method. We show that this method, which can be interpreted as a generalized Gauss-Newton method, mimics Newton’s method in function space except for an orthogonal projection onto the tangent space of the model. We show that for a variety of PDEs, natural energy gradients converge rapidly and approximations to the solution of the PDE are several orders of magnitude more accurate than gradient descent, Adam and Newton’s methods, even when these methods are given more computational time.:Chapter 1. Introduction 1 1.1 Notation and conventions 7 Part I. Geometry of Markov decision processes 11 Chapter 2. Background on Markov decision processes 12 2.1 State-action frequencies 19 2.2 The advantage function and Bellman optimality 23 2.3 Rational structure of the reward and an explicit line theorem 26 2.4 Solution methods for Markov decision processes 35 Chapter 3. State-action geometry of partially observable MDPs 44 3.1 The state-action polytope of fully observables systems 45 3.2 State-action geometry of partially observable systems 54 3.3 Number and location of critical points 69 3.4 Reward optimization in state-action space (ROSA) 83 Chapter 4. Geometry and convergence of natural policy gradient methods 94 4.1 Natural gradients 96 4.2 Natural policy gradient methods 101 4.3 Convergence of natural policy gradient flows 107 4.4 Locally quadratic convergence for regularized problems 128 4.5 Discussion and outlook 131 Part II. Neural network-based PDE solvers 133 Chapter 5. Theoretical analysis of the boundary penalty method for neural network-based PDE solvers 134 5.1 Presentation and discussion of the main results 137 5.2 Preliminaries regarding Sobolev spaces and neural networks 146 5.3 Proofs regarding uniform convergence for the deep Ritz method 150 5.4 Proofs of error estimates for the deep Ritz method 156 5.5 Proofs of implications of exact boundary values in residual minimization 167 Chapter 6. Energy natural gradients for neural network-based PDE solvers 174 6.1 Energy natural gradients 176 6.2 Experiments 183 6.3 Conclusion and outlook 192 Bibliography 193

<p>  </p> <p>A <em>digital twin</em> (DT), which keeps track of nuclear reactor history to provide real-time predictions, has been recently proposed for nuclear reactor monitoring. A digital twin can be implemented using either a differential equations-based physics model, or a data-driven machine learning model<strong>. </strong>The principal challenge in physics model-based DT consists of achieving sufficient model fidelity to represent a complex experimental system, while the main challenge in data-driven DT appears in the extensive training requirements and potential lack of predictive ability. </p> <p>In this thesis, we investigate the performance of a hybrid approach, which is based on physics-informed neural networks (PINNs) that encode fundamental physical laws into the loss function of the neural network. In this way, PINNs establish theoretical constraints and biases to supplement measurement data and provide solution to several limitations of purely data-driven machine learning (ML) models. We develop a PINN model to solve the point kinetic equations (PKEs), which are time dependent stiff nonlinear ordinary differential equations that constitute a nuclear reactor reduced-order model under the approximation of ignoring the spatial dependence of the neutron flux. PKEs portray the kinetic behavior of the system, and this kind of approach is the basis for most analyses of reactor systems, except in cases where flux shapes are known to vary with time. This system describes the nuclear parameters such as neutron density concentration, the delayed neutron precursor density concentration and reactivity. Both neutron density and delayed neutron precursor density concentrations are the vital parameters for safety and the transient behavior of the reactor power. </p> <p>The PINN model solution of PKEs is developed to monitor a start-up transient of the Purdue University Reactor Number One (PUR-1) using experimental parameters for the reactivity feedback schedule and the neutron source. The facility under modeling, PUR-1, is a pool type small research reactor located in West Lafayette Indiana. It is an all-digital light water reactor (LWR) submerged into a deep-water pool and has a power output of 10kW. The results demonstrate strong agreement between the PINN solution and finite difference numerical solution of PKEs. We investigate PINNs performance in both data interpolation and extrapolation. </p> <p>The findings of this thesis research indicate that the PINN model achieved highest performance and lowest errors in data interpolation. In the case of extrapolation data, three different test cases were considered, the first where the extrapolation is performed in a five-seconds interval, the second where the extrapolation is performed in a 10-seconds interval, and the third where the extrapolation is performed in a 15-seconds interval. The extrapolation errors are comparable to those of interpolation predictions. Extrapolation accuracy decreases with increasing time interval.</p>

A power system consists of several critical components necessary for providing electricity from the producers to the consumers. Monitoring the lifetime of power system components becomes vital since they are subjected to electrical currents and high temperatures, which affect their ageing. Estimating the component's ageing rate close to the end of its lifetime is the motivation behind our project. Knowing the ageing rate and life expectancy, we can possibly better utilize and re-utilize existing power components and their parts. In return, we could achieve better material utilization, reduce costs, and improve sustainability designs, contributing to the circular industry development of power system components. Monitoring the thermal distribution and the degradation of the insulation materials informs the estimation of the components' health state. Moreover, further study of the employed paper material of their insulation system can lead to a deeper understanding of its thermal characterization and a possible consequent improvement. Our study aims to create a model that couples the physical equations that govern the deterioration of the insulation systems of power components with modern machine learning algorithms.  As the data is limited and complex in the field of components' ageing, Physics-Informed Neural Networks (PINNs) can help to overcome the problem. PINNs exploit the prior knowledge stored in partial differential equations (PDEs) or ordinary differential equations (ODEs) modelling the involved systems. This prior knowledge becomes a regularization agent, constraining the space of available solutions and consequently reducing the training data needed.  This thesis is divided into two parts: the first focuses on the insulation system of power transformers, and the second is an exploration of the paper material concentrating on cellulose nanofibrils (CNFs) classification. The first part includes modelling the thermal distribution and the degradation of the cellulose inside the power transformer. The deterioration of one of the two systems can lead to severe consequences for the other. Both abilities of PINNs to approximate the solution of the equations and to find the parameters that best describe the data are explored. The second part could be conceived as a standalone; however, it leads to a further understanding of the paper material. Several CNFs materials and concentrations are presented, and this thesis proposes a basic unsupervised learning using clustering algorithms like k-means and Gaussian Mixture Models (GMMs) for their classification. / Ett kraftsystem består av många kritiska komponenter som är nödvändiga för att leverera el från producenter till konsumenter. Att övervaka livslängden på kraftsystemets komponenter är avgörande eftersom de utsätts för elektriska strömmar och höga temperaturer som påverkar deras åldrande. Att uppskatta komponentens åldringshastighet nära slutet av dess livslängd är motivationen bakom vårt projekt. Genom att känna till åldringshastigheten och den förväntade livslängden kan vi eventuellt utnyttja och återanvända befintliga kraftkomponenter och deras delar   bättre. I gengäld kan vi uppnå bättre materialutnyttjande, minska kostnaderna och förbättra hållbarhetsdesignen vilket bidrar till den cirkulära industriutvecklingen av kraftsystemskomponenter. Övervakning av värmefördelningen och nedbrytningen av isoleringsmaterialen indikerar komponenternas hälsotillstånd. Dessutom kan ytterligare studier av pappersmaterial i kraftkomponenternas isoleringssystem leda till en djupare förståelse av dess termiska karaktärisering och en möjlig förbättring.  Vår studie syftar till att skapa en modell som kombinerar de fysiska ekvationer som styr försämringen av isoleringssystemen i kraftkomponenter med moderna algoritmer för maskininlärning. Eftersom datan är begränsad och komplex när det gäller komponenters åldrande kan  fysikinformerade neurala nätverk (PINNs) hjälpa till att lösa problemet. PINNs utnyttjar den förkunskap som finns lagrad i partiella differentialekvationer (PDE) eller ordinära differentialekvationer (ODE) för att modellera system och använder dessa ekvationer för att begränsa antalet tillgängliga lösningar och därmed minska den mängd träningsdata som behövs.  Denna avhandling är uppdelad i två delar: den första fokuserar på krafttransformatorers isoleringssystem, och den andra är en undersökning av pappersmaterialet som används med fokus på klassificering av cellulosananofibriller (CNF). Den första delen omfattar modellering av värmefördelningen och nedbrytningen av cellulosan inuti krafttransformatorn. En försämring av ett av de två systemen kan leda till allvarliga konsekvenser för det andra. Både PINNs förmåga att approximera lösningen av ekvationerna och att hitta de parametrar som bäst beskriver datan undersöks. Den andra delen skulle kunna ses som en fristående del, men den leder till en utökad förståelse av själva pappersmaterialet. Flera CNF-material och koncentrationer presenteras och denna avhandling föreslår en simpel oövervakad inlärning med klusteralgoritmer som k-means och Gaussian Mixture Models (GMMs) för deras klassificering. / <p>QC 20231010</p>

<p>The need to carry out Uncertainty quantification (UQ) is ubiquitous in science and engineering. However, carrying out UQ for real-world problems is not straightforward and they require a lot of computational budget and resources. The objective of this thesis is to develop computationally efficient approaches based on machine learning to carry out UQ. Specifically, we addressed two problems.</p> <p><br></p> <p>The first problem is, it is difficult to carry out Uncertainty propagation (UP) in systems governed by elliptic PDEs with spatially varying uncertain fields in coefficients and boundary conditions. Here as we have functional uncertainties, the number of uncertain parameters is large. Unfortunately, in these situations to carry out UP we need to solve the PDE a large number of times to obtain convergent statistics of the quantity governed by the PDE. However, solving the PDE by a numerical solver repeatedly leads to a computational burden. To address this we proposed to learn the surrogate of the solution of the PDE in a data-free manner by utilizing the physics available in the form of the PDE. We represented the solution of the PDE as a deep neural network parameterized function in space and uncertain parameters. We introduced a physics-informed loss function derived from variational principles to learn the parameters of the network. The accuracy of the learned surrogate is validated against the corresponding ground truth estimate from the numerical solver. We demonstrated the merit of using our approach by solving UP problems and inverse problems faster than by using a standard numerical solver.</p> <p><br></p> <p>The second problem we focused on in this thesis is related to inverse problems. State of the art approach to solving inverse problems involves posing the inverse problem as a Bayesian inference task and estimating the distribution of input parameters conditioned on the observed data (posterior). Markov Chain Monte Carlo (MCMC) methods and variational inference methods provides us ways to estimate the posterior. However, these inference techniques need to be re-run whenever a new set of observed data is given leading to a computational burden. To address this, we proposed to learn a Bayesian inverse map i.e., the map from the observed data to the posterior. This map enables us to do on-the-fly inference. We demonstrated our approach by solving various examples and we validated the posteriors learned from our approach against corresponding ground truth posteriors from the MCMC method.</p>