This thesis evaluates the efficacy of Physics-Informed Neural Networks (PINNs) in simulating fluid dynamics challenges, focusing on the Burgers' equation and the lid-driven cavity problem, to develop a robust PINN framework for nuclear engineering applications such as the Sustainable Nuclear Energy Research In Sweden (SUNRISE) project. The research compares various PINN models to traditional Computational Fluid Dynamics (CFD) simulations to enhance predictive accuracy and computational efficiency for reactor design. The study analyses and optimises diverse PINN configurations, employing automatic and numerical differentiation techniques and their integrative approaches, while investigating the incorporation of advanced artificial viscosity methods to augment model robustness and address limitations of standalone PINN methods. Results show that enhanced PINN strategies achieve superior accuracy in solving the Burgers' equation and the lid-driven cavity problem at increased Reynolds numbers. For the Burgers' equation, one method with artificial viscosity achieved a Mean Squared Error (MSE) of 1.19⨉10⁻³. For the lid-driven cavity problem at Re 1000, another method without artificial viscosity yielded MSEs of 2.27⨉10⁻⁴, 9.54⨉10⁻⁵, and 1.81⨉10⁻⁵ for u, v, and p, respectively. These advancements highlight the potential of PINNs in nuclear engineering applications, particularly in tackling flow-accelerated corrosion and erosion in lead-cooled fast reactors within the SUNRISE project. / Denna avhandling utvärderar effektiviteten av Physics-Informed Neural Networks (PINNs) vid simulering av fluiddynamikutmaningar, med fokus på Burgers’ ekvation och problem med lockdriven kavitet, för att utveckla en robust PINN-ram för tillämpningar inom kärnteknik, såsom Sustainable Nuclear Energy Research i Sverige (SUNRISE) projekt. Forskningen jämför olika PINN-modeller med traditionella Computational Fluid Dynamics (CFD) simuleringar för att förbättra prediktiv noggrannhet och beräknings effektivitet för reaktordesign. Studien analyserar och optimerar olika PINN-konfigurationer, genom att använda automatiska och numeriska differentieringstekniker och deras integrerade tillvägagångssätt, samtidigt som den undersöker införandet av avancerade artificiella viskositet metoder för att öka modellens robusthet och åtgärda begränsningarna hos enskilda PINN-metoder. Resultaten visar att förbättrade PINN-strategier uppnår överlägsen noggrannhet i lösningen av Burgers’ ekvation och problem med lockdriven kavitet vid ökade Reynolds-nummer. För Burgers’ ekvation uppnådde en metod med artificiell viskositet ett medelkvadratiskt fel (MSE) på 1,19⨉10⁻³. För problem med lockdriven kavitet vid Re 1000 uppnådde en annan metod utan artificiell viskositet MSE på 2,27⨉10⁻⁴, 9,54⨉10⁻⁵ och 1,81⨉10⁻⁵ för u, v och p, respektive. Dessa framsteg framhäver potentialen hos PINNs i kärntekniska tillämpningar, särskilt i att hantera flödesaccelererad korrosion och erosion i blykylda snabba reaktorer inom SUNRISE-projektet.

In this work, I propose a hybrid particle simulator for charged particles. The simulator consists of a physics-informed neural network, which can handle arbitrary external electric fields with continuous coordinates by solving the Poisson equation, and a graph-based algorithm that computes the interacting forces between the particles. The simulator is then applied to a set of particles inside a square domain under the influence of some external electric field. As the system evolves in time, particles will gradually leave the domain causing the particle density of the domain to change. This work aims to find a model which describes the particle density evolution of the system.

Artificial neural networks (ANNs) are commonly considered as "black boxes": they can approximate any function without giving any interpretation. Novel research has observed that the laws of physics, which govern everything around us, can supplement the implementation of a neural network. For this purpose, we have physics-informed neural networks (PINNs): they are networks trained to consider the physics outlined in nonlinear partial differential equations (PDEs).  This thesis focuses on the thermal modelling of power transformers applying PINNs constrained to the heat diffusion equation. The aim is to estimate the top-oil temperature and the thermal distribution of a transformer. A solution of the equation will be provided by the Finite Volume Method (FVM), which will constitute a benchmark for the PINNs predictions. Differently from other works on PINNs, an additional challenge in this problem is the availability of field measurements. The results obtained show good accuracy in estimating the distribution and the top-oil temperature with PINN almost mimicking exactly FVM. Further improvements could be attained by rearranging the equation using more specific parameters to model the thermal behaviour of transformers and scaling the equation to dimensionless form. / Artificiella neurala nätverk (ANN) betraktas vanligtvis som "svarta lådor": de kan approximera vilken funktion som helst utan att tillhandahålla någon tolkning. Inom ny forskning har man sett att fysikens lagar, som styr allt runt omkring oss, kan komplettera implementeringen av ett neuralt nätverk. För detta ändamål har formulerats fysikinformerade neurala nätverk (PINN): de är nätverk som har tränats att ta hänsyn till den fysik som beskrivs i ickelinjära partiella differentialekvationer (PDE). Denna avhandling fokuserar på termisk modellering av transformatorer med tillämpning av PINN begränsat till värmeledningsekvationen. Syftet är att uppskatta en toppoljetemperatur och en transformators värmefördelning. Lösningen till ekvationen erhålls med finita volymmetoden (FVM), som används som en referenslösning för att utvärdera förutsägelserna från PINN. Implementeringen av PINN-algoritmen medförde en extra utmaning eftersom källtermen innefattade uppmätta värden. En metod att kringgå denna svårighet genom att approximera värdena på mätningarna i det neurala nätverket genom träning på motsvarande data presenteras. De erhållna resultaten visar god noggrannhet vid uppskattning av fördelningen och toppoljetemperaturen med PINN i jämförelse med FVM-lösningen. Ytterligare förbättringar kan uppnås genom att arrangera om ekvationen med mer specifika parametrar för att modellera transformatorernas termiska beteende och skalning av ekvationen till dimensionslös form.

Physics-Informed Neural Networks (PINNs) are hybrid models that incorporate differential equations into the training of neural networks, with the aim of bringing the best of both worlds. This project used a mathematical model describing a Continuous Stirred-Tank Reactor (CSTR), to test two possible applications of PINNs. The first type of PINN was trained to predict an unknown reaction rate law, based only on the differential equation and a time series of the reactor state. The resulting model was used inside a multi-step solver to simulate the system state over time. The results showed that the PINN could accurately model the behaviour of the missing physics also for new initial conditions. However, the model suffered from extrapolation error when tested on a larger reactor, with a much lower reaction rate. Comparisons between using a numerical derivative or automatic differentiation in the loss equation, indicated that the latter had a higher robustness to noise. Thus, it is likely the best choice for real applications. A second type of PINN was trained to forecast the system state one-step-ahead based on previous states and other known model parameters. An ordinary feed-forward neural network with an equal architecture was used as baseline. The second type of PINN did not outperform the baseline network. Further studies are needed to conclude if or when physics-informed loss should be used in autoregressive applications.

Subsurface porous media, like aquifers, petroleum reservoirs, and geothermal systems, are vital for natural resources and environmental management. Extensive research has been conducted to understand flow and transport in these media, addressing challenges in hydrocarbon extraction, carbon storage and waste management. Classifying the type of porous media (e.g., sandstone, carbonate) is often the first step in the rock characterization process, and it provides critical information regarding the physical properties of the porous media. Therefore, we utilize multivariate statistical methods with discriminant analysis to categorize porous media samples which proved to be efficient by achieving excellent classification accuracy on testing datasets and served as a surrogate tool to study key porous media characteristics. While recent advances in three-dimensional (3D) imaging of core samples have enabled digital subsurface characterization, the exorbitant computational cost associated with direct numerical simulation in 3D remains a persistent challenge. In contrast, machine learning (ML) models are much more efficient, though their use in subsurface characterization is still in its infancy. Therefore, we introduce a novel 3D convolution neural network (CNN) for end-to-end prediction of permeability. By increasing dataset size, diversity, and optimizing the network architecture, our model surpasses the accuracy of existing 3D CNN models for permeability prediction. It demonstrates excellent generalizability, accurately predicting permeability in previously unseen samples. However, despite the efficiency of the developed 3D CNN model for accurate and fast permeability prediction, its utility remains limited to small subdomains of the digital rock samples. Therefore, we introduce an upscaling technique using a new analytical solution to calculate effective permeability in a 3D digital rock composed of 2 × 2 × 2 anisotropic cells. By incorporating this solution into physics-informed neural network (PINN) models, we achieve highly accurate results. Even when upscaling previously unseen samples at multiple levels, the PINN with the physics-informed module maintains excellent accuracy. This advancement enhances the capability of ML models, like 3D CNN, for efficient and accurate digital rock analysis at the core scale. After successfully applying ML models in permeability prediction, we now extend their application to another important parameter in subsurface engineering projects: effective thermal conductivity, which is a key parameter in engineering projects like radioactive waste repositories, geothermal energy production, and underground energy storage. To address the need for large training data and processing power in ML models, we propose a novel framework based on transfer learning. This approach allows prior knowledge from previous applications to be transferred, resulting in faster and more efficient implementation of new relevant applications. We introduce CNN models trained on various porous media samples that leverage transfer learning to predict porous media sample thermal conductivity accurately. Our approach reduces training time, processing power, and data requirements, enabling effective prediction and analysis of porous media properties such as permeability and thermal conductivity. It also facilitates the application of ML to other properties, improving efficiency and accuracy. / Thesis / Doctor of Philosophy (PhD)

Probabilistic inference, especially in the Bayesian framework, is a foundation for quantifying uncertainties in geophysical inversion applications. However, due to the presence of high-dimensional datasets and the large-scale nature of geophysical inverse problems, the applicability and scalability of probabilistic inference face significant challenges for such applications. This thesis is dedicated to improving the probabilistic inference algorithms' scalability and demonstrating their applicability for large-scale geophysical inversion applications. In this thesis, I delve into three leading applied approaches in computing the Bayesian posterior distribution in geophysical inversion applications: Laplace's approximation, Markov chain Monte Carlo (MCMC), and variational Bayesian inference. The first approach, Laplace's approximation, is the simplest form of approximation for intractable Bayesian posteriors. However, its accuracy relies on the estimation of the posterior covariance matrix. I study the visualization of the misfit landscape in low-dimensional subspace and the low-rank approximations of the covariance for full waveform inversion (FWI). I demonstrate that a non-optimal Hessian's eigenvalues truncation for the low-rank approximation will affect the approximation accuracy of the standard deviation, leading to a biased statistical conclusion. Furthermore, I also demonstrate the propagation of uncertainties within the Bayesian physics-informed neural networks for hypocenter localization applications through this approach. For the MCMC approach, I develop approximate Langevin MCMC algorithms that provide fast sampling at efficient computational costs for large-scale Bayesian FWI; however, this inflates the variance due to asymptotic bias. To account for this asymptotic bias and assess their sample quality, I introduce the kernelized Stein discrepancy (KSD) as a diagnostic tool. When larger computational resources are available, exact MCMC algorithms (i.e., with a Metropolis-Hastings criterion) should be favored for an accurate posterior distribution statistical analysis. For the variational Bayesian inference, I propose a regularized variational inference framework that performs posterior inference by implicitly regularizing the Kullback-Leibler divergence loss with a deep denoiser through a Plug-and-Play method. I also developed Plug-and-Play Stein Variational Gradient Descent (PnP-SVGD), a novel algorithm to sample the regularized posterior distribution. The PnP-SVGD demonstrates its ability to produce high-resolution, trustworthy samples representative of the subsurface structures for a post-stack seismic inversion application.

<p>Extrusion Deposition Additive Manufacturing is a process by which short fiber-reinforced polymers are extruded in a screw and deposited onto a build platform using a set of instructions specified in the form of a machine code. The highly non-isothermal process can lead to undesired effects in the form of residual deformation and part delamination. Process simulations that can predict residual deformation and part delamination have been a thrust area of research to prevent the repeated trial and error process before a useful part has been produced. However, populating the material properties required for the process simulations require extensive characterization efforts. Tackling this experimental bottleneck is the focus of the first half of this research.</p><p>The first contribution is a method to infer the fiber orientation state from only tensile tests. While measuring fiber orientation state using computed tomography and optical microscopy is possible, they are often time-consuming, and limited to measuring fibers with circular cross-sections. The knowledge of the fiber orientation is extremely useful in populating material properties using micromechanics models. To that end, two methods to infer the fiber orientation state are proposed. The first is Bayesian methodology which accounts for aleatoric and epistemic uncertainty. The second method is a deterministic method that returns an average value of the fiber orientation state and polymer properties. The inferred orientation state is validated by performing process simulations using material properties populated using the inferred orientation state. A different challenge arises when dealing with multiple extrusion systems. Considering even the same material printed on different extrusion systems requires an engineer to redo the material characterization efforts (due to changes in microstructure). This, in turn, makes characterization efforts expensive and time-consuming. Therefore, the objective of the second contribution is to address this experimental bottleneck and use prior information about the material manufactured in one extrusion system to predict its properties when manufactured in another system. A framework that can transfer thermal conductivity data while accounting for uncertainties arising from different sources is presented. The predicted properties are compared to experimental measurements and are found to be in good agreement.</p><p>While the process simulations using finite element methods provide a reliable framework for the prediction of residual deformation and part delamination, they are often computationally expensive. Tackling the fundamental challenges regarding this computational bottleneck is the focus of the second half of this dissertation. To that end, as the third contribution, a neural network based solver is developed that can solve parametric partial differential equations. This is attained by deriving the weak form of the governing partial differential equation. Using this variational form, a novel loss function is proposed that does not require the evaluation of the integrals arising out of the weak form using Gauss quadrature methods. Rather, the integrals are identified to be expectation values for which an unbiased estimator is developed. The method is tested for parabolic and elliptical partial differential equations and the results compare well with conventional solvers. Finally, the fourth contribution of this dissertation involves using the new solver to solve heat transfer problems in additive manufacturing, without the need for discretizing the time domain. A neural network is used to solve the governing equations in the evolving geometry. The weak form based loss is altered to account for the evolving geometry by using a novel sequential collocation sampling method. This work forms the foundational work to solve parametric problems in additive manufacturing.</p>

<p>Nuclear engineering hosts diverse domains including, but not limited to, power plant automation, human-machine interfacing, detection and identification of special nuclear materials, modeling of reactor kinetics and dynamics that most frequently are described by systems of differential equations (DEs), either ordinary (ODEs) or partial ones (PDEs). In this work we study multiple problems related to safety and Special Nuclear Material detection, and numerical solutions for partial differential equations using neural networks. More specifically, this work is divided in six chapters. Chapter 1 is the introduction, in Chapter</p> <p>2 we discuss the development of a gamma-ray radionuclide library for the characterization</p> <p>of gamma-spectra. In Chapter 3, we present a new approach, the ”Variance Counterbalancing”, for stochastic</p> <p>large-scale learning. In Chapter 4, we introduce a systematic approach for constructing proper trial solutions to partial differential equations (PDEs) of up to second order, using neural forms that satisfy prescribed initial, boundary and interface conditions. Chapter 5 is about an alternative, less imposing development of neural-form trial solutions for PDEs, inside rectangular and non-rectangular convex boundaries. Chapter 6 presents an ensemble method that avoids the multicollinearity issue and provides</p> <p>enhanced generalization performance that could be suitable for handling ”few-shots”- problems frequently appearing in nuclear engineering.</p>

<p dir="ltr">The adoption of Heterogeneous Integration (HI) technologies in semiconductor packaging to build 2.5D/3D structures has led to increased power densities and material heterogeneity. These structures place a new burden on thermal and mechanical design. Additionally, these structures allow for significantly increased physical design freedom. With more possible layout options as well as tougher thermal constraints, new specialized tools are required to accelerate this type of design.</p><p dir="ltr">To address this problem using traditional finite element analysis Stack3D is presented. Stack3D is a steady-state thermal-mechanical geometry modeling and analysis platform for advanced packaging early design exploration. It is a finite element simulator developed from scratch in Matlab complete with symbolic geometry representation, automatic meshing, chip power map support, and sparse matrix acceleration.</p><p dir="ltr">After the development of Stack3D, methods for further accelerating the simulation process at the cost of solution accuracy were examined. Neural networks were selected as an engine for this task based on their millisecond evaluation time. In order to choose between the training paradigms of Physics Informed and Data Driven neural networks, a series of benchmarks were run to identify Data Driven networks as ideal candidates for steady state heat conduction.</p><p dir="ltr">Last, the first neural network model for fully generalized steady state heat conduction of 3D packages is developed. This is made possible by using the solution to the partial differential equation governing steady state heat conduction and casting the problem into an image-to-image translation framework. After accounting for the third spatial dimension, this allows the use of cutting edge image processing network for the heat conduction problem. After training, the network was able to run tens of thousands of simulations with an average of 0.53\% error and 0.0035 seconds per simulation.</p>