The goal of this work is to improve the robustness and generalization of deep learning models, using a similar approach to the unsupervised "innate learning" strategy in visual development. A series of research studies are presented to demonstrate how an unsupervised machine learning efficient coding approach can create filters similar to the receptive fields of the primary visual cortex (V1) in the brain, and these filters are capable of pretraining convolutional neural networks (CNNs) to enable faster training times and higher accuracy with less dependency on the source data. Independent component analysis (ICA) is used for unsupervised feature extraction as it has shown success in both applied machine learning and modeling biological neural receptive fields. This pretraining applies equally well to various forms of visual input, including natural color images, black and white, binocular, and video to drive the V1-like Gabor filters in the brain. For efficient processing of typical visual scenes, the filters that ICA produces are developed by encoding natural images. These filters are then used to initialize the kernels in the first layer of a CNN to train on the CIFAR-10 dataset to perform image classification. Results show that the ICA initialization for a custom made CNN produces models with a test accuracy up to 12% better than the standard model in the first 10 epochs, which for specific accuracy thresholds reduces the number of training epochs by approximately 40% (to reach 60% accuracy) and 50% (to reach 70% accuracy). Additionally, this pre-training results in marginally higher accuracy even after extensive training over 50 epochs. This proposed method of unsupervised machine learning to pre-train weights in deep learning improves both training time and accuracy, which is why it is observed in biological networks and is finding increased application in applied deep learning.
| Identifer | oai:union.ndltd.org:unt.edu/info:ark/67531/metadc1986488 |
| Date | 08 1900 |
| Creators | Behpour, Sahar |
| Contributors | Albert, Mark, Xiao, Ting, Hawamdeh, Suliman, Grigolini, Paolo |
| Publisher | University of North Texas |
| Source Sets | University of North Texas |
| Language | English |
| Detected Language | English |
| Type | Thesis or Dissertation |
| Format | Text |
| Rights | Public, Behpour, Sahar, Copyright, Copyright is held by the author, unless otherwise noted. All rights Reserved. |
Page generated in 0.002 seconds