No description available.

No description available.

No description available.

No description available.

Smart manufacturing is a revolutionary domain integrating advanced sensing technology, machine learning methods, and the industrial internet of things (IIoT). The development of sensing technology provides large amounts and various types of data (e.g., profile, image, point cloud, etc.) to describe each stage of a manufacturing process. The machine learning methods have the advantages of efficiently and effectively processing and fusing large-scale datasets and demonstrating outstanding performance in different tasks (e.g., diagnosis, monitoring, etc.). Despite the advantages of incorporating machine learning methods into smart manufacturing, there are some widely existing concerns in practice: (1) Most of the edge devices in the manufacturing system only have limited memory space and computational capacity; (2) Both the performance and interpretability of the data analytics method are desired; (3) The connection to the internet exposes the manufacturing system to cyberattacks, which decays the trustiness of data, models, and results. To address these limitations, this dissertation proposed systematic engineering-driven machine learning methods to improve the system intelligence for smart manufacturing. The contributions of this dissertation can be summarized in three aspects. First, tensor decomposition is incorporated to approximately compress the convolutional (Conv) layer in Deep Neural Network (DNN), and a novel layer is proposed accordingly. Compared with the Conv layer, the proposed layer significantly reduces the number of parameters and computational costs without decaying the performance. Second, a physics-informed stochastic surrogate model is proposed by incorporating the idea of building and solving differential equations into designing the stochastic process. The proposed method outperforms pure data-driven stochastic surrogates in recovering system patterns from noised data points and exploiting limited training samples to make accurate predictions and conduct uncertainty quantification. Third, a Wasserstein-based out-of-distribution detection (WOOD) framework is proposed to strengthen the DNN-based classifier with the ability to detect adversarial samples. The properties of the proposed framework have been thoroughly discussed. The statistical learning bound of the proposed loss function is theoretically investigated. The proposed framework is generally applicable to DNN-based classifiers and outperforms state-of-the-art benchmarks in identifying out-of-distribution samples. / Doctor of Philosophy / The global industries are experiencing the fourth industrial revolution, which is characterized by the use of advanced sensing technology, big data analytics, and the industrial internet of things (IIoT) to build a smart manufacturing system. The massive amount of data collected in the engineering process provides rich information to describe the complex physical phenomena in the manufacturing system. The big data analytics methods (e.g., machine learning, deep learning, etc.) are developed to exploit the collected data to complete specific tasks, such as checking the quality of the product, diagnosing the root cause of defects, etc. Given the outstanding performances of the big data analytics methods in these tasks, there are some concerns arising from the engineering practice, such as the limited available computational resources, the model's lack of interpretability, and the threat of hacking attacks. In this dissertation, we propose systematic engineering-driven machine learning methods to address or mitigate these widely existing concerns. First, the model compression technique is developed to reduce the number of parameters and computational complexity of the deep learning model to fit the limited available computational resources. Second, physics principles are incorporated into designing the regression method to improve its interpretability and enable it better explore the properties of the data collected from the manufacturing system. Third, the cyberattack detection method is developed to strengthen the smart manufacturing system with the ability to detect potential hacking threats.

Tsunami and storm surge are two of the main destructive and costly natural hazards faced by coastal communities around the world. To enhance coastal resilience and to develop effective risk management strategies, accurate and efficient tsunami and storm surge prediction models are needed. However, existing physics-based numerical models have the disadvantage of being difficult to satisfy both accuracy and efficiency at the same time. In this dissertation, several surrogate models are developed using statistical and machine learning techniques that can rapidly predict a tsunami and storm surge without substantial loss of accuracy, with respect to high-fidelity physics-based models. First, a tsunami run-up response function (TRRF) model is developed that can rapidly predict a tsunami run-up distribution from earthquake fault parameters. This new surrogate modeling approach reduces the number of simulations required to build a surrogate model by separately modeling the leading order contribution and the residual part of the tsunami run-up distribution. Secondly, a TRRF-based inversion (TRRF-INV) model is developed that can infer a tsunami source and its impact from tsunami run-up records. Since this new tsunami inversion model is based on the TRRF model, it can perform a large number of tsunami forward simulations in tsunami inversion modeling, which is impossible with physics-based models. And lastly, a one-dimensional convolutional neural network combined with principal component analysis and k-means clustering (C1PKNet) model is developed that can rapidly predict the peak storm surge from tropical cyclone track time series. Because the C1PKNet model uses the tropical cyclone track time series, it has the advantage of being able to predict more diverse tropical cyclone scenarios than the existing surrogate models that rely on a tropical cyclone condition at one moment (usually at or near landfall). The surrogate models developed in this dissertation have the potential to save lives, mitigate coastal hazard damage, and promote resilient coastal communities. / Doctor of Philosophy / Tsunami and storm surge can cause extensive damage to coastal communities; to reduce this damage, accurate and fast computer models are needed that can predict the water level change caused by these coastal hazards. The problem is that existing physics-based computer models are either accurate but slow or less accurate but fast. In this dissertation, three new computer models are developed using statistical and machine learning techniques that can rapidly predict a tsunami and storm surge without substantial loss of accuracy compared to the accurate physics-based computer models. Three computer models are as follows: (1) A computer model that can rapidly predict the maximum ground elevation wetted by the tsunami along the coastline from earthquake information, (2) A computer model that can reversely predict a tsunami source and its impact from the observations of the maximum ground elevation wetted by the tsunami, (3) A computer model that can rapidly predict peak storm surges across a wide range of coastal areas from the tropical cyclone's track position over time. These new computer models have the potential to improve forecasting capabilities, advance understanding of historical tsunami and storm surge events, and lead to better preparedness plans for possible future tsunamis and storm surges.

Large scale Gaussian process (GP) regression is infeasible for larger data sets due to cubic scaling of flops and quadratic storage involved in working with covariance matrices. Remedies in recent literature focus on divide-and-conquer, e.g., partitioning into sub-problems and inducing functional (and thus computational) independence. Such approximations can speedy, accurate, and sometimes even more flexible than an ordinary GPs. However, a big downside is loss of continuity at partition boundaries. Modern methods like local approximate GPs (LAGPs) imply effectively infinite partitioning and are thus pathologically good and bad in this regard. Model averaging, an alternative to divide-and-conquer, can maintain absolute continuity but often over-smooth, diminishing accuracy. Here I propose putting LAGP-like methods into a local experts-like framework, blending partition-based speed with model-averaging continuity, as a flagship example of what I call precision aggregated local models (PALM). Using N_C LAGPs, each selecting n from N data pairs, I illustrate a scheme that is at most cubic in n, quadratic in N_C, and linear in N, drastically reducing computational and storage demands. Extensive empirical illustration shows how PALM is at least as accurate as LAGP, can be much faster in terms of speed, and furnishes continuous predictive surfaces. Finally, I propose sequential updating scheme which greedily refines a PALM predictor up to a computational budget, and several variations on the basic PALM that may provide predictive improvements. / Doctor of Philosophy / Occasionally, when describing the relationship between two variables, it may be helpful to use a so-called ``non-parametric" regression that is agnostic to the function that connects them. Gaussian Processes (GPs) are a popular method of non-parametric regression used for their relative flexibility and interpretability, but they have the unfortunate drawback of being computationally infeasible for large data sets. Past work into solving the scaling issues for GPs has focused on ``divide and conquer" style schemes that spread the data out across multiple smaller GP models. While these model make GP methods much more accessible to large data sets they do so either at the expense of local predictive accuracy of global surface continuity. Precision Aggregated Local Models (PALM) is a novel divide and conquer method for GP models that is scalable for large data while maintaining local accuracy and a smooth global model. I demonstrate that PALM can be built quickly, and performs well predictively compared to other state of the art methods. This document also provides a sequential algorithm for selecting the location of each local model, and variations on the basic PALM methodology.

This research is a proof-of-concept study to develop a surrogate model, using deep learning (DL), to predict solution error for a given model with a given mesh. For this research, we have taken the von Mises stress contours and have predicted two different types of error indicators contours, namely (i) von Mises error indicator (MISESERI), and (ii) energy density error indicator (ENDENERI). Error indicators are designed to identify the solution domain areas where the gradient has not been properly captured. It uses the spatial gradient distribution of the existing solution for a given mesh to estimate the error. Due to poor meshing and nature of the finite element method, these error indicators are leveraged to study and reduce errors in the finite element solution using an adaptive remeshing scheme. Adaptive re-meshing is an iterative and computationally expensive process to reduce the error computed during the post-processing step. To overcome this limitation we propose an approach to replace it using data-driven techniques. We have introduced an image processing-based surrogate model designed to solve an image-to-image regression problem using convolutional neural networks (CNN) that takes a 256 × 256 colored image of von mises stress contour and outputs the required error indicator. To train this model with good generalization performance we have developed four different geometries for each of the three case studies: (i) quarter plate with a hole, (b) simply supported plate with multiple holes, and (c) simply supported stiffened plate. The entire research is implemented in a three phase approach, phase I involves the design and development of a CNN to perform training on stress contour images with their corresponding von Mises stress values volume-averaged over the entire domain. Phase II involves developing a surrogate model to perform image-to-image regression and the final phase III involves extending the capabilities of phase II and making the surrogate model more generalized and robust. The final surrogate model used to train the global dataset of 12,000 images consists of three auto encoders, one encoder-decoder assembly, and two multi-output regression neural networks. With the error of less than 1% in the neural network training shows good memorization and generalization performance. Our final surrogate model takes 15.5 hours to train and less than a minute to predict the error indicators on testing datasets. Thus, this present study can be considered a good first step toward developing an adaptive remeshing scheme using deep neural networks. / Master of Science / This research is a proof-of-concept study to develop an image processing-based neural network (NN) model to solve an image-to-image regression problem. In finite element analysis (FEA), due to poor meshing and nature of the finite element method, these error indicators are used to study and reduce errors. For this research, we have predicted two different types of error indicator contours by using stress images as inputs to the NN model. In popular FEA packages, adaptive remeshing scheme is used to optimize mesh quality by iteratively computing error indicators making the process computationally expensive. To overcome this limitation we propose an approach to replace it using convolutional neural networks (CNN). Such neural networks are particularly used for image based data. To train our CNN model with good generalization performance we have developed four different geometries with varying load cases. The entire research is implemented in a three phase approach, phase I involves the design and development of a CNN model to perform initial level training on small image size. Phase II involves developing an assembled neural network to perform image-to-image regression and the final phase III involves extending the capabilities of phase II for more generalized and robust results. With the error of less than 1% in the neural network training shows good memorization and generalization performance. Our final surrogate model takes 15.5 hours to train and less than a minute to predict the error indicators on testing datasets. Thus, this present study can be considered a good first step toward developing an adaptive remeshing scheme using deep neural networks.

<p dir="ltr">Neural networks (NNs) effectively address the challenges of engineering design optimization by using data-driven models, thus reducing computational demands. However, their effectiveness depends heavily on hyperparameter optimization (HPO), which is a global optimization problem. While traditional HPO methods, such as manual, grid, and random search, are simple, they often fail to navigate the vast hyperparameter (HP) space efficiently. This work examines the effectiveness of integrating Bayesian optimization (BO) with multi-armed bandit (MAB) optimization for HPO in NNs. The thesis initially addresses HPO in one-shot sampling, where NNs are trained using datasets of varying sample sizes. It compares the performance of NNs optimized through traditional HPO techniques and a combination of BO and MAB optimization on the analytical Branin function and aerodynamic shape optimization (ASO) of an airfoil in transonic flow. Findings from the optimization of the Branin function indicate that the combined BO and MAB optimization approach leads to simpler NNs and reduces the sample size by approximately 10 to 20 compared to traditional HPO methods, all within half the time. This efficiency improvement is even more pronounced in ASO, where the BO and MAB optimization use about 100 fewer samples than the traditional methods to achieve the optimized airfoil design. The thesis then expands on adaptive HPOs within the framework of efficient global optimization (EGO) using a NN-based prediction and uncertainty (EGONN) algorithm. It employs the BO and MAB optimization for tuning HPs during sequential sampling, either every iteration (HPO-1itr) or every five iterations (HPO-5itr). These strategies are evaluated against the EGO as a benchmark method. Through experimentation with the analytical three-dimensional Hartmann function and ASO, assessing both comprehensive and selective tunable HP sets, the thesis contrasts adaptive HPO approaches with a static HPO method (HPO-static), which uses the initial HP settings throughout. Initially, a comprehensive set of the HPs is optimized and evaluated, followed by an examination of selectively chosen HPs. For the optimization of the three-dimensional Hartmann function, the adaptive HPO strategies surpass HPO-static in performance in both cases, achieving optimal convergence and sample efficiency comparable to EGO. In ASO, applying the adaptive HPO strategies reduces the baseline airfoil's drag coefficient to 123 drag counts (d.c.) for HPO-1itr and 120 d.c. for HPO-5itr when tuning the full set of the HPs. For a selected subset of the HPs, 123 d.c. and 121 d.c. are achieved by HPO-1itr and HPO-5itr, respectively, which are comparable to the minimum achieved by EGO. While the HPO-static method reduces the drag coefficient to 127 d.c. by tuning a subset of the HPs, which is a 15 d.c. reduction from its full set case, it falls short of the minimum of adaptive HPO strategies. Focusing on a subset of the HPs reduces time costs and enhances the convergence rate without sacrificing optimization efficiency. The time reduction is more significant with higher HPO frequencies as HPO-1itr cuts time by 66%, HPO-5itr by 38%, and HPO-static by 2%. However, HPO-5itr still requires 31% of the time needed by HPO-1itr for the full HP tuning and 56% for the subset HP tuning.</p>

The application of stochastic modeling for learning the behavior of multibody dynamics models is investigated. The stochastic modeling technique is also known as Kriging or random function approach. Post-processing data from a simulation run is used to train the stochastic model that estimates the relationship between model inputs, such as the suspension relative displacement and velocity, and the output, for example, sum of suspension forces. Computational efficiency of Multibody Dynamics (MBD) models can be improved by replacing their computationally-intensive subsystems with stochastic predictions. The stochastic modeling technique is able to learn the behavior of a physical system and integrate its behavior in MBS models, resulting in improved real-time simulations and reduced computational effort in models with repeated substructures (for example, modeling a train with a large number of rail vehicles). Since the sampling plan greatly influences the overall accuracy and efficiency of the stochastic predictions, various sampling plans are investigated, and a space-filling Latin Hypercube sampling plan based on the traveling salesman problem (TPS) is suggested for efficiently representing the entire parameter space. The simulation results confirm the expected increased modeling efficiency, although further research is needed for improving the accuracy of the predictions. The prediction accuracy is expected to improve through employing a sampling strategy that considers the discrete nature of the training data and uses infill criteria that considers the shape of the output function and detects sample spaces with high prediction errors. It is recommended that future efforts consider quantifying the computation efficiency of the proposed learning behavior by overcoming the inefficiencies associated with transferring data between multiple software packages, which proved to be a limiting factor in this study. These limitations can be overcome by using the user subroutine functionality of SIMPACK and adding the stochastic modeling technique to its force library. / Ph. D.