No description available.

Transformer neural networks have made a significant impact on natural language processing. The Transformer network self-attention mechanism effectively addresses the vanishing gradient problem that limits a network learning capability, especially when the time series gets longer or the size of the network gets deeper. This dissertation examines the usage of the Transformer model for time-series forecasting and customizes it for a simultaneous multistep-ahead prediction model in a surrogate model predictive control (MPC) application. The proposed method demonstrates enhanced control performance and computation efficiency compared to the Long-short term memory (LSTM)-based MPC and one-step-ahead prediction model structures for both LSTM and Transformer networks. In addition to the Transformer, this research investigates hybrid machine-learning modeling. The machine learning models are known for superior function approximation capability with sufficient data. However, the quantity and quality of data to ensure the prediction precision are usually not readily available. The physics-informed neural network (PINN) is a type of hybrid modeling method using dynamic physics-based equations in training a standard machine learning model as a form of multi-objective optimization. The PINN approach with the state-of-the-art time-series neural networks Transformer is studied in this research providing the standard procedure to develop the Physics-Informed Transformer (PIT) and validating with various case studies. This research also investigates the benefit of nonlinear model-based control and estimation algorithms for managed pressure drilling (MPD). This work presents a new real-time high-fidelity flow model (RT-HFM) for bottom-hole pressure (BHP) regulation in MPD operations. Lastly, this paper presents details of an Arduino microcontroller temperature control lab as a benchmark for modeling and control methods. Standard benchmarks are essential for comparing competing models and control methods, especially when a new method is proposed. A physical benchmark considers real process characteristics such as the requirement to meet a cycle time, discrete sampling intervals, communication overhead with the process, and model mismatch. Novel contributions of this work are (1) a new MPC system built upon a Transformer time-series architecture, (2) a training method for time-series machine learning models that enables multistep-ahead prediction, (3) verification of Transformer MPC solution time performance improvement (15 times) over LSTM networks, (4) physics-informed machine learning to improve extrapolation potential, and (5) two case studies that demonstrate hybrid modeling and benchmark performance criteria.

Orbit propagation is the backbone of many problems in the space domain, such as uncertainty quantification, trajectory optimization, and guidance, navigation, and control of on orbit vehicles. Many of these techniques can rely on millions of orbit propagations, slowing computation, especially evident on low-powered satellite hardware. Past research has relied on the use of lookup tables or data streaming to enable on orbit solutions. These solutions prove inaccurate or ineffective when communication is interrupted. In this work, we introduce the use of physics-informed neural networks (PINNs) for orbit propagation to achieve fast and accurate on-board solutions, accelerated by GPU hardware solutions now available in satellite hardware. Physics-informed neural networks leverage the governing equations of motion in network training, allowing the network to optimize around the physical constraints of the system. This work leverages the use of unsupervised learning and introduces the concept of fundamental integrals of orbits to train PINNs to solve orbit problems with no knowledge of the true solution. Numerical experiments are conducted for both Earth orbits and cislunar space, being the first time a neural network integrator is implemented on flight-like hardware. The results show that the use of PINNs can decrease solution evaluation time by several order of magnitude while retaining accurate solutions to the perturbed two-body problem and the circular restricted three-body problem for deployment on spaceflight-like hardware. Implementation of these neural networks aim to reduce computational time to allow for real-time evaluation of complex algorithms on-board space vehicles.

The component-level performance of materials is fundamentally determined by the underlying microstructural features. Therefore, designing high-performance materials using multi-scale models plays a significant role to improve the predictability, reliability, proper functioning, and longevity of components for a wide range of applications in the fields of aerospace, electronics, energy, and structural engineering. This thesis aims to develop new methodologies to design microstructures under inherent material uncertainty by incorporating machine learning techniques. To achieve this objective, the study addresses gradient-based and machine learning-driven design optimization methods to enhance homogenized linear and non-linear properties of polycrystalline microstructures. However, variations arising from the thermo-mechanical processing of materials affect microstructural features and properties by propagating over multiple length scales. To quantify this inherent microstructural uncertainty, this study introduces a linear programming-based analytical method. When this analytical uncertainty quantification formulation is not applicable (e.g., uncertainty propagation on non-linear properties), a machine learning-based inverse design approach is presented to quantify the microstructural uncertainty. Example design problems are discussed for different polycrystalline systems (e.g., Titanium, Aluminium, and Galfenol). Though conventional machine learning performs well when used for designing microstructures or modeling material properties, its predictions may still fail to satisfy design constraints associated with the physics of the system. Therefore, the physics-informed neural network (PINN) is developed to incorporate problem physics in the machine learning formulation. In this study, a PINN model is built and integrated into materials design to study the deformation processes of Copper and a Titanium-Aluminum alloy. / Doctor of Philosophy / Microstructure-sensitive design is a high-throughput computational approach for materials design, where material performance is improved through the control and design of microstructures. It enhances component performance and, subsequently, the overall system's performance at the application level. This thesis aims to design microstructures for polycrystalline materials such as Galfenol, Titanium-Aluminum alloys, and Copper to obtain desired mechanical properties for certain applications. The advantage of the microstructure-sensitive design approach is that multiple microstructures can be suggested, which provide a similar value of the design parameters. Therefore, manufacturers can follow any of these microstructure designs to fabricate the materials with the desired properties. Moreover, the microstructure uncertainty arising from the variations in thermo-mechanical processing and measurement of the experimental data is quantified. It is necessary to address the resultant randomness of the microstructure because it can alter the expected mechanical properties. To check the manufacturability of proposed microstructure designs, a physics-informed machine learning model is developed to build a relation between the process, microstructure, and material properties. This model can be used to solve the process design problem to identify the processing parameters to achieve a given/desired microstructure.

Probabilistic inference, especially in the Bayesian framework, is a foundation for quantifying uncertainties in geophysical inversion applications. However, due to the presence of high-dimensional datasets and the large-scale nature of geophysical inverse problems, the applicability and scalability of probabilistic inference face significant challenges for such applications. This thesis is dedicated to improving the probabilistic inference algorithms' scalability and demonstrating their applicability for large-scale geophysical inversion applications. In this thesis, I delve into three leading applied approaches in computing the Bayesian posterior distribution in geophysical inversion applications: Laplace's approximation, Markov chain Monte Carlo (MCMC), and variational Bayesian inference. The first approach, Laplace's approximation, is the simplest form of approximation for intractable Bayesian posteriors. However, its accuracy relies on the estimation of the posterior covariance matrix. I study the visualization of the misfit landscape in low-dimensional subspace and the low-rank approximations of the covariance for full waveform inversion (FWI). I demonstrate that a non-optimal Hessian's eigenvalues truncation for the low-rank approximation will affect the approximation accuracy of the standard deviation, leading to a biased statistical conclusion. Furthermore, I also demonstrate the propagation of uncertainties within the Bayesian physics-informed neural networks for hypocenter localization applications through this approach. For the MCMC approach, I develop approximate Langevin MCMC algorithms that provide fast sampling at efficient computational costs for large-scale Bayesian FWI; however, this inflates the variance due to asymptotic bias. To account for this asymptotic bias and assess their sample quality, I introduce the kernelized Stein discrepancy (KSD) as a diagnostic tool. When larger computational resources are available, exact MCMC algorithms (i.e., with a Metropolis-Hastings criterion) should be favored for an accurate posterior distribution statistical analysis. For the variational Bayesian inference, I propose a regularized variational inference framework that performs posterior inference by implicitly regularizing the Kullback-Leibler divergence loss with a deep denoiser through a Plug-and-Play method. I also developed Plug-and-Play Stein Variational Gradient Descent (PnP-SVGD), a novel algorithm to sample the regularized posterior distribution. The PnP-SVGD demonstrates its ability to produce high-resolution, trustworthy samples representative of the subsurface structures for a post-stack seismic inversion application.

<p>Extrusion Deposition Additive Manufacturing is a process by which short fiber-reinforced polymers are extruded in a screw and deposited onto a build platform using a set of instructions specified in the form of a machine code. The highly non-isothermal process can lead to undesired effects in the form of residual deformation and part delamination. Process simulations that can predict residual deformation and part delamination have been a thrust area of research to prevent the repeated trial and error process before a useful part has been produced. However, populating the material properties required for the process simulations require extensive characterization efforts. Tackling this experimental bottleneck is the focus of the first half of this research.</p><p>The first contribution is a method to infer the fiber orientation state from only tensile tests. While measuring fiber orientation state using computed tomography and optical microscopy is possible, they are often time-consuming, and limited to measuring fibers with circular cross-sections. The knowledge of the fiber orientation is extremely useful in populating material properties using micromechanics models. To that end, two methods to infer the fiber orientation state are proposed. The first is Bayesian methodology which accounts for aleatoric and epistemic uncertainty. The second method is a deterministic method that returns an average value of the fiber orientation state and polymer properties. The inferred orientation state is validated by performing process simulations using material properties populated using the inferred orientation state. A different challenge arises when dealing with multiple extrusion systems. Considering even the same material printed on different extrusion systems requires an engineer to redo the material characterization efforts (due to changes in microstructure). This, in turn, makes characterization efforts expensive and time-consuming. Therefore, the objective of the second contribution is to address this experimental bottleneck and use prior information about the material manufactured in one extrusion system to predict its properties when manufactured in another system. A framework that can transfer thermal conductivity data while accounting for uncertainties arising from different sources is presented. The predicted properties are compared to experimental measurements and are found to be in good agreement.</p><p>While the process simulations using finite element methods provide a reliable framework for the prediction of residual deformation and part delamination, they are often computationally expensive. Tackling the fundamental challenges regarding this computational bottleneck is the focus of the second half of this dissertation. To that end, as the third contribution, a neural network based solver is developed that can solve parametric partial differential equations. This is attained by deriving the weak form of the governing partial differential equation. Using this variational form, a novel loss function is proposed that does not require the evaluation of the integrals arising out of the weak form using Gauss quadrature methods. Rather, the integrals are identified to be expectation values for which an unbiased estimator is developed. The method is tested for parabolic and elliptical partial differential equations and the results compare well with conventional solvers. Finally, the fourth contribution of this dissertation involves using the new solver to solve heat transfer problems in additive manufacturing, without the need for discretizing the time domain. A neural network is used to solve the governing equations in the evolving geometry. The weak form based loss is altered to account for the evolving geometry by using a novel sequential collocation sampling method. This work forms the foundational work to solve parametric problems in additive manufacturing.</p>

<p>Nuclear engineering hosts diverse domains including, but not limited to, power plant automation, human-machine interfacing, detection and identification of special nuclear materials, modeling of reactor kinetics and dynamics that most frequently are described by systems of differential equations (DEs), either ordinary (ODEs) or partial ones (PDEs). In this work we study multiple problems related to safety and Special Nuclear Material detection, and numerical solutions for partial differential equations using neural networks. More specifically, this work is divided in six chapters. Chapter 1 is the introduction, in Chapter</p> <p>2 we discuss the development of a gamma-ray radionuclide library for the characterization</p> <p>of gamma-spectra. In Chapter 3, we present a new approach, the ”Variance Counterbalancing”, for stochastic</p> <p>large-scale learning. In Chapter 4, we introduce a systematic approach for constructing proper trial solutions to partial differential equations (PDEs) of up to second order, using neural forms that satisfy prescribed initial, boundary and interface conditions. Chapter 5 is about an alternative, less imposing development of neural-form trial solutions for PDEs, inside rectangular and non-rectangular convex boundaries. Chapter 6 presents an ensemble method that avoids the multicollinearity issue and provides</p> <p>enhanced generalization performance that could be suitable for handling ”few-shots”- problems frequently appearing in nuclear engineering.</p>