Global ETD Search

Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.

Results

Showing 1 to 20 of 60 for “"Batch Normalization"”.

  1. Batch Normalization Preconditioning for Neural Network Training

    Batch normalization (BN) is a popular and ubiquitous method in deep learning that has been shown to decrease training time and improve generalization performance of neural networks. Despite its success, BN is not theoretically well understood. It is not suitable for use with very small mini-batch …

    uky-thes Repository record for Batch Normalization Preconditioning for Neural Network Training (opens in a new tab)

  2. Orthogonal Recurrent Neural Networks and Batch Normalization in Deep Neural Networks

    … this work focuses on the often used technique of batch normalization. Although found to be successful in decreasing training times and in preventing overfitting, it is still unknown why this technique works. In this paper we describe batch normalization and provide a potential alternative with the …

    uky-thes Repository record for Orthogonal Recurrent Neural Networks and Batch Normalization in Deep Neural Networks (opens in a new tab)

  3. Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization

    … In the thesis, the limitation of conventional batch normalization layer on adversarial learning based domain adaptation methods is mentioned and discussed. From the view of the limitation, we propose replacing the Sharing Affine Transformation with our proposed Separate Affine Transformation …

    ottawa-retro Repository record for Domain Adaptation on Semantic Segmentation with Separate Affine Transformation in Batch Normalization (opens in a new tab)

  4. Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks

    … on training dynamics, focusing specifically on normalization. To this end, we study batch normalization, a widely-used technique that has proven to accelerate training and improve generalization in deep neural networks. Despite its popularity, our understanding of its underlying mechanisms …

    ubc Repository record for Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks (opens in a new tab) Download PDF for Towards understanding self-attention and batch normalization : optimization and margin dynamics in neural networks (opens in a new tab)

  5. Quantifying the Performance and Resource Usage of HLS4ML's Implementation of the Batch Normalization Layer on FPGAs

    … synthesis for machine learning, specifically the batch normalization layer, seeking to evaluate the quality, resource usage, and performance of the models produced using this technique. Our research indicates that HLS designs are efficient but not entirely accurate, whereas the optimized …

    washington Repository record for Quantifying the Performance and Resource Usage of HLS4ML's Implementation of the Batch Normalization Layer on FPGAs (opens in a new tab)

  6. Normalization Techniques for Sequential and Graphical Data

    Normalization methods have proven to be an invaluable tool in the training of deep neural networks. In particular, Layer and Batch Normalization are commonly used to mitigate the risks of exploding and vanishing gradients. This work presents two methods which are related to these normalization …

    uky-thes Repository record for Normalization Techniques for Sequential and Graphical Data (opens in a new tab)

  7. A Convolutional Neural Network Approach to Breast Cancer Tumor Boundary Detection

    … of two lightweight models, with and without batch normalization and dropout layers, were compared for accuracy, loss, and processing times. The model including batch normalization and dropout layers outperformed the network without these functions; the enhanced model was additionally compared …

    calpoly Repository record for A Convolutional Neural Network Approach to Breast Cancer Tumor Boundary Detection (opens in a new tab)

  8. Mathematical Guarantees of Near-Optimal Cross-Entropy Loss through Neural Collapse

    … target classes, neural network classifiers with batch normalization and weight decay achieve intra-class feature cosine similarity near one and inter-class cosine similarity near $-\frac{1}{C-1}$, which justifies multiple observations of Neural Collapse under realistic conditions guaranteed by …

    gatech Repository record for Mathematical Guarantees of Near-Optimal Cross-Entropy Loss through Neural Collapse (opens in a new tab)

  9. Performance Comparison of Binarized Neural Network with Convolutional Neural Network

    … The factors like Dropout rate, Dataset size, Batch Normalization layer, Filter size, and Dropout layer addition has been studied on CNN. But there is a lack of literature in the study of dropout and the various factors in Binarized Neural Network. This report will provide a brief introduction …

    mtu Repository record for Performance Comparison of Binarized Neural Network with Convolutional Neural Network (opens in a new tab)

  10. Describing Images by Semantic Modeling using Attributes and Tags

    … the characteristics and behavior of the Batch Normalization. We conclude this dissertation by providing a fresh view, in light of information geometry and Fisher kernels to why the batch normalization works. We propose Mixture Normalization that disentangles modes of variation in the …

    ucf Repository record for Describing Images by Semantic Modeling using Attributes and Tags (opens in a new tab)

  11. Computational Methods for MRI-Guided and Powered Ferric Applicators: Modeling and Image Processing

    … from short-axis cine MRI images. BNU-net is the batch normalized (BN) U-net, LNU-net is the layer normalized (LN) U-net, and IBU-net is the instance-batch normalized (IB) U-net. The architectures of BNU-net, LNU-net, and IBU-net have an encoding path for feature extraction and a decoding path …

    houston Repository record for Computational Methods for MRI-Guided and Powered Ferric Applicators: Modeling and Image Processing (opens in a new tab)

  12. Gradient Conditioning in Deep Neural Networks

    … in the weights of the network when each batch gradient is estimated and differences between the input values in each batch. Some architectural traits, like skip-connections and batch-normalization, allow much deeper networks to be trained by reducing each type of variance and improving …

    byu Repository record for Gradient Conditioning in Deep Neural Networks (opens in a new tab) Download PDF for Gradient Conditioning in Deep Neural Networks (opens in a new tab)

  13. Slimmable neural networks for edge devices

    … we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, …

    uiuc Repository record for Slimmable neural networks for edge devices (opens in a new tab)

  14. REGULARIZATION ON MACHINE LEARNING

    … verifies its capability to be used together with Batch Normalization (Ioffe and Szegedy 2015). We perform Drop-Activation on the benchmark datasets and show that the performance of popular networks can be improved generally by Drop-Activation.

    nus Repository record for REGULARIZATION ON MACHINE LEARNING (opens in a new tab) Download file for REGULARIZATION ON MACHINE LEARNING (opens in a new tab)

  15. Flexi-Compression: A Flexible Model Compression Method for Autonomous Driving

    … the original model. In addition, we use a Flexi-Batch Normalization layer to prune the model and finally further reduce the model size by model quantization. We compress an autonomous driving algorithm and achieve excellent performance.

    carleton Repository record for Flexi-Compression: A Flexible Model Compression Method for Autonomous Driving (opens in a new tab)

  16. IMPROVED TRAINING OF DEEP NETWORKS FOR COMPUTER VISION

    … step-size selection. We propose alternative “big batch” SGD schemes that adaptively grow the batch size over time to maintain a nearly constant signal-to-noise ratio in the gradient approximation. The high fidelity gradients enable automated learning rate selection and do not require stepsize …

    maryland Repository record for IMPROVED TRAINING OF DEEP NETWORKS FOR COMPUTER VISION (opens in a new tab)

  17. Improving Deep Neural Networks: Optimization, Regularization, and Generative Modeling

    … correlation between features. We then exploit batch normalization to efficiently remove the correlation effect. Finally, we focus on generative modeling, a fundamental application of DNNs. We propose a framework for training autoencoder-based generative models, with non-adversarial losses and …

    calgary Repository record for Improving Deep Neural Networks: Optimization, Regularization, and Generative Modeling (opens in a new tab)

  18. Anytime Computer Vision for Autonomous Driving

    … multiple input resolutions through perresolution batch normalization, enabling runtime selection of an ideal input scale under a latency budget, as well as resolution retargeting, which allows multi-resolution training without access to the original training dataset. We implement and evaluate our …

    ku Repository record for Anytime Computer Vision for Autonomous Driving (opens in a new tab)

  19. Pruning and quantization for efficient deep neural networks

    … our approach by taking into account both the batch-normalization layers and activation functions. In this way, it is possible to train deep neural networks that can be evaluated without floating-point operations after training.<br><br>Next, we propose a novel filter pruning method that is …

    freiburg-diss Repository record for Pruning and quantization for efficient deep neural networks (opens in a new tab)

  20. An investigation into the use of ConvNext within IICS/IIDS framework for person Re-ID

    … techniques such as Adaptive Instance and Batch Normalization (AIBN) and Transform Normalization (TNorm) to minimize intra-camera and inter-camera variations respectively, our work emphasizes the application of ConvNeXt as a feature extractor. ConvNeXt gets higher mAP and CMC on the …

    uoit Repository record for An investigation into the use of ConvNext within IICS/IIDS framework for person Re-ID (opens in a new tab)

Page 1 of 3