Despite the impressive performance of deep neural networks (DNNs) on numerous vision tasks, they still exhibit yet-to-understand uncouth behaviours. One puzzling behaviour is the reaction of DNNs to various noise attacks, where it has been shown that there exist small adversarial noise that can result in a severe degradation in the performance of DNNs. To rigorously treat this, we derive exact analytic expressions for the first and second moments (mean and variance) of a small piecewise linear (PL) network with a single rectified linear unit (ReLU) layer subject to general Gaussian input. We experimentally show that these expressions are tight under simple linearizations of deeper PL-DNNs, especially popular architectures in the literature (e.g. LeNet and AlexNet). Extensive experiments on image classification show that these expressions can be used to study the behaviour of the output mean of the logits for each class, the inter-class confusion and the pixel-level spatial noise sensitivity of the network. Moreover, we show how these expressions can be used to systematically construct targeted and non-targeted adversarial attacks. Then, we proposed a special estimator DNN, named mixture of linearizations (MoL), and derived the analytic expressions for its output mean and variance, as well. We employed these expressions to train the model to be particularly robust against Gaussian attacks without the need for data augmentation. Upon training this network on a loss that is consolidated with the derived output probabilistic moments, the network is not only robust under very high variance Gaussian attacks but is also as robust as networks that are trained with 20 fold data augmentation.

This study presents the results of an extraction of the 2nd-order nonlinear responses from model test data. Emphasis is given on the effects of assumptions made for the Gaussian and non-Gaussian input on the estimation of the 2nd-order response, employing the quadratic Volterra model. The effects of sea severity and data length on the estimation of response are also investigated at the same time. The data sets used in this study are surge forces on a fixed barge, a surge motion of a compliant mini TLP (Tension Leg Platform), and surge forces on a fixed and truncated column. Sea states are used from rough sea (Hs=3m) to high sea (Hs=9m) for a barge case, very rough sea (Hs=3.9m) for a mini TLP, and phenomenal sea (Hs=15m) for a truncated column. After the estimation of the response functions, the outputs are reconstructed and the 2nd order nonlinear responses are extracted with all the QTF distributed in the entire bifrequency domain. The reconstituted time series are compared with the experiment in both the time and frequency domains. For the effects of data length on the estimation of the response functions, 3, 15, and 40- hour data were investigated for a barge, but 3-hour data was used for a mini TLP and a fixed and truncated column due to lack of long data. The effects of sea severity on the estimation of the response functions are found in both methods. The non-Gaussian method for estimation is more affected by data length than the Gaussian method.

No description available.