Yes / Weight initialization of neural networks has an important influence on the learning process, and the selection of initial weights is related to the activation interval of the activation function. It is proposed that an improved and extended weight initialization method for neural network with asymmetric activation function as an extension of the linear interval tolerance method (LIT), called ‘GLIT’ (generalized LIT), which is more suitable for higher-dimensional inputs. The purpose is to expand the selection range of the activation function so that the input falls in the unsaturated region, so as to improve the performance of the network. Then, a tolerance solution theorem based upon neural network system is given and proved. Furthermore, the algorithm is given about determining the initial weight interval. The validity of the theorem and algorithm is verified by numerical experiments. The input could fall into any preset interval in the sense of probability under the GLIT method. In another sense, the GLIT method could provide a theoretical basis for the further study of neural networks. / The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This paper is partly supported by National Science Foundation of China under Grants (62073226, 61603262), Liaoning Province Natural Science Foundation (2020-KF-11-09, 2021-KF-11-05), Shen-Fu Demonstration Zone Science and Technology Plan Project (2020JH13, 2021JH07), Central Government Guides Local Science and Technology Development Funds of Liaoning Province (2021JH6).

The goal of this work is to improve the robustness and generalization of deep learning models, using a similar approach to the unsupervised "innate learning" strategy in visual development. A series of research studies are presented to demonstrate how an unsupervised machine learning efficient coding approach can create filters similar to the receptive fields of the primary visual cortex (V1) in the brain, and these filters are capable of pretraining convolutional neural networks (CNNs) to enable faster training times and higher accuracy with less dependency on the source data. Independent component analysis (ICA) is used for unsupervised feature extraction as it has shown success in both applied machine learning and modeling biological neural receptive fields. This pretraining applies equally well to various forms of visual input, including natural color images, black and white, binocular, and video to drive the V1-like Gabor filters in the brain. For efficient processing of typical visual scenes, the filters that ICA produces are developed by encoding natural images. These filters are then used to initialize the kernels in the first layer of a CNN to train on the CIFAR-10 dataset to perform image classification. Results show that the ICA initialization for a custom made CNN produces models with a test accuracy up to 12% better than the standard model in the first 10 epochs, which for specific accuracy thresholds reduces the number of training epochs by approximately 40% (to reach 60% accuracy) and 50% (to reach 70% accuracy). Additionally, this pre-training results in marginally higher accuracy even after extensive training over 50 epochs. This proposed method of unsupervised machine learning to pre-train weights in deep learning improves both training time and accuracy, which is why it is observed in biological networks and is finding increased application in applied deep learning.

We consider a mechanistic non-linear machine learning approach to learning signals in financial time series data. A modularised and decoupled algorithm framework is established and is proven on daily sampled closing time-series data for JSE equity markets. The input patterns are based on input data vectors of data windows preprocessed into a sequence of daily, weekly and monthly or quarterly sampled feature measurement changes (log feature fluctuations). The data processing is split into a batch processed step where features are learnt using a Stacked AutoEncoder (SAE) via unsupervised learning, and then both batch and online supervised learning are carried out on Feedforward Neural Networks (FNNs) using these features. The FNN output is a point prediction of measured time-series feature fluctuations (log differenced data) in the future (ex-post). Weight initializations for these networks are implemented with restricted Boltzmann machine pretraining, and variance based initializations. The validity of the FNN backtest results are shown under a rigorous assessment of backtest overfitting using both Combinatorially Symmetrical Cross Validation and Probabilistic and Deflated Sharpe Ratios. Results are further used to develop a view on the phenomenology of financial markets and the value of complex historical data under unstable dynamics.