<p>Artificial
neural networks, including deep neural networks, play a central role in data-driven
science due to their superior learning capacity and adaptability to different
tasks and data structures. However, although quantitative uncertainty analysis
is essential for training and deploying reliable data-driven models, the
uncertainties in neural networks are often overlooked or underestimated in many
studies, mainly due to the lack of a high-fidelity and computationally
efficient uncertainty quantification approach. In this work, a novel
uncertainty analysis scheme is developed. The Gaussian mixture model is used to
characterize the probability distributions of uncertainties in arbitrary forms,
which yields higher fidelity than the presumed distribution forms, like
Gaussian, when the underlying uncertainty is multimodal, and is more compact
and efficient than large-scale Monte Carlo sampling. The fidelity of the
Gaussian mixture is refined through adaptive scheduling of the width of each
Gaussian component based on the active assessment of the factors that could
deteriorate the uncertainty representation quality, such as the nonlinearity of
activation functions in the neural network. </p>
<p>Following
this idea, an adaptive Gaussian mixture scheme of nonlinear uncertainty
propagation is proposed to effectively propagate the probability distributions of
uncertainties through layers in deep neural networks or through time in
recurrent neural networks. An adaptive Gaussian mixture filter (AGMF) is then designed
based on this uncertainty propagation scheme. By approximating the dynamics of
a highly nonlinear system with a feedforward neural network, the adaptive Gaussian
mixture refinement is applied at both the state prediction and Bayesian update
steps to closely track the distribution of unmeasurable states. As a result,
this new AGMF exhibits state-of-the-art accuracy with a reasonable
computational cost on highly nonlinear state estimation problems subject to
high magnitudes of uncertainties. Next, a probabilistic neural network with
Gaussian-mixture-distributed parameters (GM-PNN) is developed. The adaptive
Gaussian mixture scheme is extended to refine intermediate layer states and
ensure the fidelity of both linear and nonlinear transformations within the
network so that the predictive distribution of output target can be inferred
directly without sampling or approximation of integration. The derivatives of the
loss function with respect to all the probabilistic parameters in this network
are derived explicitly, and therefore, the GM-PNN can be easily trained with
any backpropagation method to address practical data-driven problems subject to
uncertainties.</p>
<p>The
GM-PNN is applied to two data-driven condition monitoring schemes of
manufacturing processes. For tool wear monitoring in the turning process, a
systematic feature normalization and selection scheme is proposed for the
engineering of optimal feature sets extracted from sensor signals. The
predictive tool wear models are established using two methods, one is a type-2
fuzzy network for interval-type uncertainty quantification and the other is the
GM-PNN for probabilistic uncertainty quantification. For porosity monitoring in
laser additive manufacturing processes, convolutional neural network (CNN) is
used to directly learn patterns from melt-pool patterns to predict porosity.
The classical CNN models without consideration of uncertainty are compared with
the CNN models in which GM-PNN is embedded as an uncertainty quantification
module. For both monitoring schemes, experimental results show that the GM-PNN
not only achieves higher prediction accuracies of process conditions than the
classical models but also provides more effective uncertainty quantification to
facilitate the process-level decision-making in the manufacturing environment.</p><p>Based
on the developed uncertainty analysis methods and their proven successes in
practical applications, some directions for future studies are suggested.
Closed-loop control systems may be synthesized by combining the AGMF with
data-driven controller design. The AGMF can also be extended from a state estimator
to the parameter estimation problems in data-driven models. In addition, the
GM-PNN scheme may be expanded to directly build more complicated models like
convolutional or recurrent neural networks.</p>
| Identifer | oai:union.ndltd.org:purdue.edu/oai:figshare.com:article/15153588 |
| Date | 12 August 2021 |
| Creators | Bin Zhang (11073474) |
| Source Sets | Purdue University |
| Detected Language | English |
| Type | Text, Thesis |
| Rights | CC BY 4.0 |
| Relation | https://figshare.com/articles/thesis/Data-driven_Uncertainty_Analysis_in_Neural_Networks_with_Applications_to_Manufacturing_Process_Monitoring/15153588 |
Page generated in 0.0025 seconds