Global ETD Search
Search theses and dissertations gathered from participating repositories worldwide. Every result links back to the library that holds it. No account is needed.
Results
Showing 1 to 20 of 24 for “"structured pruning"”.
-
Compression and Analysis of Pre-trained Language Model using Neural Slimming
… we study how to compress the BERT model in a structured pruning manner. We proposed the neural slimming technique to assess the importance of each neuron and designed the cost function and pruning strategy to remove neurons that make zero or less contribution to the prediction. After getting …
-
Methods for Optimizing BERT Model on Edge Devices - Accelerating Biomedical NLP with Pruned and Quantized BERT Models
… pruned with a three-stage, Fisher-guided structured pruning workflow at three levels of sparsity. Subsequently, in the second stage, dynamic 8-bit integers quantization using ONNX Runtime was applied to standard BERT, BioBERT, and DistilBERT. The third stage involved combining both pruning …
-
Pruning resnet neural networks block by block
Neural network pruning has gained popularity for deep models with the goal of reducing storage and computational requirements, both for inference and training. Pruning weights from neural networks to make models sparse has been discussed for a long time starting with in 1990s. Structured pruning is …
-
Optimization Strategies for Enhancing Resource Efficiency in Transformers & Large Language Models
… quantization, knowledge distillation, and pruning, focusing on energy and computational efficiency while retaining performance. Among standalone methods, 4-Bit quantization significantly reduces energy use with minimal accuracy loss. Hybrid approaches, like NVIDIA’s Minitron approach …
-
Efficient model adaptation and compression for edge intelligence
… The proposed framework integrates quantization, structured pruning, and efficient transfer learning techniques to achieve compact, adaptive, and privacy-preserving AI models. Through this joint approach, advanced AI systems can be customized and deployed efficiently on affordable edge devices, …
-
LightMARL : smart swarm coordination in urban spaces
… framework utilizes neural network quantization, structured pruning, and knowledge distillation tailored for multi-agent policy optimization (MAPPO). These methods decrease model size and computational demands while maintaining essential coordination behaviors. To enhance communication efficiency, …
-
Modeling Cognition by Pruning and Topography-Learning in Deep Neural Networks
… category. Technically, we implement this using structured pruning over learned feature maps or units in convolutional neural networks to select a subset that can improve the alignment with human judgments. This part consists of three studies. In Study 1.1, we introduce a statistic quantifying …
-
Computation-Efficient Deep Learning Models for Computer Vision and Multimodal Vision-Language Tasks via Network Pruning
… explored model compression techniques such as pruning, quantization, and distillation. These approaches aim to enhance model efficiency without compromising performance, particularly when adapting to domain-specific tasks under limited resources. This dissertation investigates several …
-
Novel Norms for Pruning Convolutional Neural Networks
… are grouped into objects called filters, and structured pruning is the process of removing a proportion of filters from a CNN without significantly reducing its accuracy. The manner in which filters are chosen affects the performance of the model. One method for selecting which filters to …
-
Mitigating effects of data heterogeneity in decentralized learning and addressing challenges of adapting foundation models
… we develop a novel search space design involving structured pruning and parameter prioritization to transform SAM into a weight-sharing supernetwork. This enables the discovery of efficient subnetworks that perform comparably to the pre-trained SAM model.
-
Resource-Aware and Label-Efficient Learning for Smart Buildings: Deployable Energy Disaggregation and Few-Shot Thermal Occupancy Estimation
… we develop a graph-aware, architecture-agnostic structured pruning engine for microcontroller-deployable energy disaggregation. Pruning during training under an activation-weighted Taylor criterion, it removes 13-73% of parameters across six architectures and three public datasets while improving …
-
Efficient Deep Neural Network Pruning: From Convolutional Networks to Large Language Models
Neural network pruning is a widely used compression technique. However, minimal performance degradation can make pruning resource-intensive, requiring substantial computational and memory overhead. It becomes particularly challenging when resources are limited. This thesis delves into the …
-
On Compressing Deep Neural Networks
… Octave Convolutional Networks with in-parallel pruning-quantization on different frequencies. Second, we propose a novel unstructured pruning pipeline, Attention-based Simultaneous sparse structure and Weight Learning (ASWL), where an efficient algorithm is proposed to calculate the pruning …
-
Optimization of battery AI models for edge deployment: A systematic study of model compression techniques and hardware-aware efficiency
… transfer, numerical equivalence verification, pruning and quantization evaluation, Simulink integration, and embedded C++ code generation. Two complementary compression directions are evaluated. Structured pruning reduces the number of learnable parameters while retaining useful predictive …
-
Towards Efficient Privacy-Preserving Deep Learning: HE-Friendly Structures, Flexible Pruning, HE-Efficient Architectures, and Secure Transformer Token Drop
… learning efficiency: First, Hunter introduces structured pruning using HE-friendly features—internal/external structure and weight diagonal—to reduce HE operations like Perm, Mult, and Add while maintaining accuracy. For instance, on VGG-16 (ImageNet), Hunter cuts Perm to 2% and Mult/Add to …
-
Deep Neural Network Pruning and Sensor Fusion in Practical 2D Detection
… their time and resource intensive. Measures like pruning and fusion, which aim to simplify the structure and lessen the load on the network’s resources, should be considered to resolve this problem. In this thesis, we intend to explore the effect of pruning on segmentation and object detection as …
-
Exploiting compression techniques for efficient edge AI
… of DNN model compression techniques: a) pruning, methods that remove weights from DNN models; b) quantization, methods that reduce the bit length of weight and/or activation values of DNN models; and c) knowledge distillation, methods that can distill the knowledge from a larger model to …
-
Co-architecting scalable intelligence: from hardware substrates to algorithms
… resources, ENLighten co-designs a PTC-aware structured pruning flow with a sparse photonic engine, enabling up to 50% parameter reduction on Vision Transformers with negligible accuracy loss. To mitigate hybrid interface bottlenecks, Fuse and Mix fuses nonlinear activation with A/D conversion …
-
Optimizing machine learning models for autonomous vehicles
… The first contribution, R-TOSS, is a novel semi-structured pruning framework for 2D ODs. R-TOSS outperforms various state-of-the-art model optimization techniques while also improving performance on embedded resource-constrained platforms. For accelerating 3D ODs, we propose UPAQ, which uses a …
-
Efficient, Robust, and Self-Supervised Algorithm-Hardware Co-Design
… as a starting point, this research presents a structured pruning algorithm followed by the dedicated in-memory-computing deployment scheme. While the efficiency has been improved, the non-ideality of the individual ReRAM device and non-reliable retention characteristics destroy the accuracy of …
Page 1 of 2