Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 60 for “"Batch Normalization"”.
-
Batch Normalization Preconditioning for Neural Network Training
Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks. Despite its success, BN is not theoretically well understood. It is not suitable for use with very small mini-batch …
-
Orthogonal Recurrent Neural Networks and Batch Normalization in Deep Neural Networks
… this work focuses on the often used technique of batch normalization. Although found to be successful in decreasing training times and in preventing overfitting, it is still unknown why this technique works. In this paper we describe batch normalization and provide a potential alternative with the …
-
Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization
… In the thesis, the limitation of conventional batch normalization layer on adversarial learning based domain adaptation methods is mentioned and discussed. From the view of the limitation, we propose replacing the Sharing Affine Transformation with our proposed Separate Affine Transformation …
-
Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks
… on training dynamics, focusing specifically on normalization. To this end, we study batch normalization, a widely-used technique that has proven to accelerate training and improve generalization in deep neural networks. Despite its popularity, our understanding of its underlying mechanisms …
-
Quantifying the Performance and Resource Usage of HLS4ML's Implementation of the Batch Normalization Layer on FPGAs
… synthesis for machine learning, specifically the batch normalization layer, seeking to evaluate the quality, resource usage, and performance of the models produced using this technique. Our research indicates that HLS designs are efficient but not entirely accurate, whereas the optimized …
-
Normalization Techniques for Sequential and Graphical Data
Normalization methods have proven to be an invaluable tool in the training of deep neural networks. In particular, Layer and Batch Normalization are commonly used to mitigate the risks of exploding and vanishing gradients. This work presents two methods which are related to these normalization …
-
A Convolutional Neural Network Approach to Breast Cancer Tumor Boundary Detection
… of two lightweight models, with and without batch normalization and dropout layers, were compared for accuracy, loss, and processing times. The model including batch normalization and dropout layers outperformed the network without these functions; the enhanced model was additionally compared …
-
Mathematical Guarantees of Near-Optimal Cross-Entropy Loss through Neural Collapse
… target classes, neural network classifiers with batch normalization and weight decay achieve intra-class feature cosine similarity near one and inter-class cosine similarity near $-\frac{1}{C-1}$, which justifies multiple observations of Neural Collapse under realistic conditions guaranteed by …
-
Performance Comparison of Binarized Neural Network with Convolutional Neural Network
… The factors like Dropout rate, Dataset size, Batch Normalization layer, Filter size, and Dropout layer addition has been studied on CNN. But there is a lack of literature in the study of dropout and the various factors in Binarized Neural Network. This report will provide a brief introduction …
-
Describing Images by Semantic Modeling using Attributes and Tags
… the characteristics and behavior of the Batch Normalization. We conclude this dissertation by providing a fresh view, in light of information geometry and Fisher kernels to why the batch normalization works. We propose Mixture Normalization that disentangles modes of variation in the …
-
Computational Methods for MRI-Guided and Powered Ferric Applicators: Modeling and Image Processing
… from short-axis cine MRI images. BNU-net is the batch normalized (BN) U-net, LNU-net is the layer normalized (LN) U-net, and IBU-net is the instance-batch normalized (IB) U-net. The architectures of BNU-net, LNU-net, and IBU-net have an encoding path for feature extraction and a decoding path …
-
Gradient Conditioning in Deep Neural Networks
… in the weights of the network when each batch gradient is estimated and differences between the input values in each batch. Some architectural traits, like skip-connections and batch-normalization, allow much deeper networks to be trained by reducing each type of variance and improving …
-
Slimmable neural networks for edge devices
… we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, …
-
REGULARIZATION ON MACHINE LEARNING
… verifies its capability to be used together with Batch Normalization (Ioffe and Szegedy 2015). We perform Drop-Activation on the benchmark datasets and show that the performance of popular networks can be improved generally by Drop-Activation.
-
Flexi-Compression: A Flexible Model Compression Method for Autonomous Driving
… the original model. In addition, we use a Flexi-Batch Normalization layer to prune the model and finally further reduce the model size by model quantization. We compress an autonomous driving algorithm and achieve excellent performance.
-
IMPROVED TRAINING OF DEEP NETWORKS FOR COMPUTER VISION
… step-size selection. We propose alternative “big batch” SGD schemes that adaptively grow the batch size over time to maintain a nearly constant signal-to-noise ratio in the gradient approximation. The high fidelity gradients enable automated learning rate selection and do not require stepsize …
-
Improving Deep Neural Networks: Optimization, Regularization, and Generative Modeling
… correlation between features. We then exploit batch normalization to efficiently remove the correlation effect. Finally, we focus on generative modeling, a fundamental application of DNNs. We propose a framework for training autoencoder-based generative models, with non-adversarial losses and …
-
Anytime Computer Vision for Autonomous Driving
… multiple input resolutions through perresolution batch normalization, enabling runtime selection of an ideal input scale under a latency budget, as well as resolution retargeting, which allows multi-resolution training without access to the original training dataset. We implement and evaluate our …
-
Pruning and quantization for efficient deep neural networks
… our approach by taking into account both the batch-normalization layers and activation functions. In this way, it is possible to train deep neural networks that can be evaluated without floating-point operations after training.<br><br>Next, we propose a novel filter pruning method that is …
-
An investigation into the use of ConvNext within IICS/IIDS framework for person Re-ID
… techniques such as Adaptive Instance and Batch Normalization (AIBN) and Transform Normalization (TNorm) to minimize intra-camera and inter-camera variations respectively, our work emphasizes the application of ConvNeXt as a feature extractor. ConvNeXt gets higher mAP and CMC on the …
Page 1 of 3