Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 11 of 11 for “"Neural Collapse"”.
-
Mathematical Guarantees of Near-Optimal Cross-Entropy Loss through Neural Collapse
Neural Collapse is a recently observed geometric structure that emerges in the final layer of neural network classifiers. Specifically, Neural Collapse states that at the terminal phase of neural networks training, 1) the intra-class variability of last layer features tends to zero, 2) the class …
-
Towards Open World Classification Using Neural Collapse Properties and Dempster–Shafer Theory
… This work proposes a novel framework combining Neural Collapse properties with Dempster-Shafer Theory (DST) to improve image classification, enable uncertainty quantification, and enhance out-of-distribution detection. Our evaluations across CIFAR-10 and SVHN test datasets demonstrate that the …
-
Learning from imbalanced data : a geometric study on over-parameterized models
… justification for the empirical finding known as Neural-collapse. Limited to the balanced setting, Neural-collapse suggests that over-parameterized models learn features and classifiers that are arranged in a perfectly symmetric and maximally-separated geometry. Leveraging the UFM in our analysis, …
-
QUANTIZATION AND APPROXIMATION OF NEURAL NETWORKS
… specific focus on the theoretical properties of neural networks. The first part of this work addresses neural network quantization. Despite the widespread use of quantization as a primary method for compressing deep neural networks, a majority of existing approaches lack provable guarantees. By …
-
Inductive bias from layerwise structure and backpropagation in neural networks: analysis through simplified models and empirical frameworks
<p>Deep Neural Networks (DNNs) are revolutionizing industries, pushing the boundaries of image and language processing. Despite their remarkable performance, we still have little understanding of how they achieve it. Unlocking their potential and ensuring their safe use requires exploring their …
-
Explaining Deep Neural Networks using Information Theory and Geometry
Abstract Deep neural networks (DNNs) have achieved remarkable success in various machine learning tasks, yet a satisfactory explanation for their generalisation performance remains elusive. In this thesis, we stand on the shoulders of giants and explore two extensions to the information bottleneck …
-
A Geometric and Topological View on Deep Learning Language Models
… for the first time, the occurrence of neural collapse in NLP models for text classification tasks, revealing how features within each class collapse towards their mean while maintaining separation between different classes. Our geometric analysis, using convex hull computations and …
-
Group Robustness under the Microscope: How Class Imbalance Shapes the Effectiveness of Last-Layer Retraining
… examination of group robustness in fine-tuned neural networks, focusing on how class imbalance and data composition shape the effectiveness of last-layer retraining (LLR) and related balancing techniques. Through extensive experiments on four benchmark datasets spanning vision and language …
-
Symmetric implicit geometries under label-imbalances : from cross-entropy to supervised contrastive loss
… learning is unclear. Recent studies reveal the Neural Collapse (NC) phenomenon in over-parameterized networks, where features from penultimate layers collapse into class-representative vectors. These vectors align with last layer classifiers, forming symmetric geometry with maximal class …
-
Explainability and generalisation properties of artificial deep neural networks
… two foundational questions about modern deep neural networks: (i) how to make their predictions explainable, and (ii) how to predict and understand their generalisation performance. We develop new theory and algorithms on formal methods for interpretability using computational complexity …
-
Analyzing training dynamics of deep neural networks: insights and limitations of the neural tangent kernel regime
The widespread use of Deep Neural Networks (DNNs) in various application has underscored their effectiveness, yet the fundamental principles behind their success largely remain elusive. Despite being highly overparametrized, DNNs often exhibit effective generalization, defying predictions of …