Aristotle University Of Thessaloniki (AUTH)
Compression and acceleration of machine learning algorithms based on low rank factorization
Abstract
dc:descriptionThe rapid proliferation of deep neural networks (DNNs) across a wide range of applications has been accompanied by a steady increase in model complexity, posing significant challenges for deployment on resource-constrained edge devices. These devices, such as wearables and embedded systems, impose strict limitations on memory footprint, computational budget, and energy consumption, making the compression and acceleration of DNNs a critical research problem. While numerous compression techniques have been proposed in the literature, including new decomposition algorithms and pruning criteria, this thesis operates at the methodology level. It proposes systematic, practical frameworks for effectively applying existing low-rank factorization (LRF) and pruning techniques in real-world DNN deployment scenarios. All methodologies operate entirely in the post-training phase, requiring no retraining from scratch and only minimal fine-tuning. LRF decomposes weight matrices and tensors into lower-rank approximations, reducing both the number of parameters and the number of floating-point operations (FLOPs). However, its practical adoption remains challenging due to the difficulty of selecting appropriate decomposition algorithms, determining optimal ranks per layer, handling inter-layer interactions, and balancing trade-offs between compression ratio, accuracy, and inference efficiency. This thesis addresses these challenges through four complementary methodologies, each targeting a different layer type or compression strategy, and together forming a comprehensive framework for practical DNN compression on edge devices. The first methodology targets fully connected (FC) layers, which are the dominant source of parameters in many DNN architectures. In its base approach, a design space exploration (DSE) methodology is proposed that systematically evaluates candidate LRF solutions based on tensor train (TT) decomposition. The extended approach additionally supports singular value decomposition (SVD), selecting the most suitable algorithm for each layer, and considers the number of parameters, FLOPs, and inference execution time on the target device. A key contribution is the identification of trade-offs between decomposition algorithms in terms of memory footprint and inference execution time, together with a systematic reduction of the design space that limits the number of candidates proceeding to fine-tuning. Rather than a single solution, the methodology provides a set of suitable LRF configurations, allowing practitioners to select the most appropriate one for each model and deployment target according to their specific constraints. The second methodology addresses FC and convolutional layers jointly, which introduces additional complexity due to the spatial structure of convolutional layers and their sensitivity to rank selection. A novel approach is proposed that accounts for inter-layer interactions by leveraging the similarity of feature maps, rather than weights, between the original and the factorized models. This enables accurate layer-wise rank selection across both layer types with minimal fine-tuning. The methodology is further extended to support multiple decomposition algorithms simultaneously, allowing each layer to be factorized with the most suitable algorithm. This extends the single-method approach, in which one decomposition algorithm is used for all layers of the same type, to a multi-method (hybrid) approach that produces compressed models with superior compression-accuracy trade-offs. Results of the multi-method (hybrid) approach on 17 convolutional neural network (CNN) models across nine datasets, including ImageNet-scale models, demonstrate average parameter reductions of 82.4% in convolutional layers and 92.7% in FC layers when minimizing the parameter count, with a relative accuracy drop of less than 1.5%, and a one-shot fine-tuning process that is, on average, 8 times faster than iterative calibration. Deployment of models optimized for FLOPs reduction on the NVIDIA Jetson AGX Xavier platform achieves average inference speedups of 2.34 times on the central processing unit (CPU) and 1.37 times on the graphics processing unit (GPU). The third methodology addresses structured pruning, specifically filter pruning in convolutional layers and neuron pruning in FC layers, as a self-adaptive extension of the inter-layer interaction framework. It identifies effective pruning configurations while minimizing search overhead, achieving compression ratios competitive with existing approaches at a significantly lower search cost: the search is up to 55 times faster than layer-wise iterative pruning and up to 35 times faster than Qualcomm's AI Model Efficiency Toolkit (AIMET). This demonstrates that, beyond compression ratio, search efficiency is itself a critical practical consideration for real-world deployment. The fourth methodology moves beyond individual compression techniques and systematically investigates the joint application of LRF, pruning (filter-based and unstructured), and post-training quantization (PTQ). It reveals that a careful ordering and combination of these techniques yield compression gains beyond what any single technique achieves in isolation, with the best-performing pipeline achieving up to 43% more parameter reduction than LRF alone and up to 75% more than filter-based pruning alone, while PTQ proves fully compatible with both. All proposed methodologies are fully parameterized, allowing practitioners to adapt them to different models, architectures, and application-specific constraints without modifying the underlying framework. They are validated on multiple DNN architectures and target platforms, covering both CPU and GPU execution, with evaluation of accuracy, energy efficiency, and inference speedup. The contributions of this thesis collectively advance the state of the art in practical DNN compression, offering a flexible and extensible framework that bridges the gap between high-level model optimization and real-world deployment on edge devices.
Degree
thesis:*- Grantor dc:publisher
- Aristotle University Of Thessaloniki (AUTH)
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Kokhazadeh, Milad
- Κοχαζαντέχ, Μιλάντ
Subjects
dc:subject × 16- Παραγοντοποίηση χαμηλής τάξης
- Κλάδεμα νευρωνικών δικτύων
- Συμπίεση βαθιών νευρωνικών δικτύων
- Συσκευές περιορισμένων πόρων
- Συσκευές άκρου δικτύου
- Low rank factorization
- Neural network pruning
- Deep neural network compression
- Resource-Constrained Devices
- Edge devices
- Επιστήμη Ηλεκτρονικών Υπολογιστών και Πληροφορική
- Φυσικές Επιστήμες
- Τεχνητή νοημοσύνη
- Computer and Information Sciences
- Natural Sciences
- Artificial Intelligence
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/63047