Democritus University of Thrace (DUTH)
Data parallel hardware accelerators for machine learning algorithms
Abstract
dc:descriptionMachine learning (ML) has rapidly evolved into a transformative technology, driving advances across diverse domains such as computer vision, natural language processing, and robotics. However, the computational demands of modern deep learning models far exceed the capabilities of conventional processors. This challenge has motivated the development of domain-specific accelerators, particularly systolic arrays, which excel at the dense and regular matrix operations that dominate neural network workloads. Achieving high performance and energy efficiency on modern models, however, also requires addressing challenges such as sparsity, fault tolerance, and workload irregularity. This thesis tackles these challenges by proposing a set of hardware optimizations that improve the performance, efficiency, and reliability of systolic arrays and their tensor-based extensions for a broad range of machine learning applications. First, we investigate microarchitectural and physical-design-oriented optimizations for dense systolic arrays. We introduce ArrayFlex, an architecture featuring a configurable pipeline that dynamically adjusts its depth across convolutional layers to minimize latency and energy consumption during convolutional neural network execution. Complementing this, we propose an asymmetric Floorplanning methodology that accounts for the inherent imbalance in horizontal and vertical interconnect demands, resulting in measurable reductions in interconnect power. Building on these foundations, we further lower switching activity through dynamic encoding and weight reordering, leveraging workload-aware data characteristics to achieve significant power savings with minimal hardware overhead. Second, we enhance the microarchitecture of sparse systolic tensor arrays designed to execute pruned machine learning models, which are often employed to reduce memory and storage requirements. We propose DeMM, a disaggregated matrix multiplication engine that supports relaxed sparsity patterns while maintaining hardware regularity, and DSSTA, a tensor array featuring a diagonal-permuted dataflow that eliminates skewing overhead. Both architectures achieve substantial reductions in latency and power consumption for modern CNN workloads. To further ensure reliability, we introduce two complementary fault-tolerance mechanisms: a checksum-based ABFT method adapted for structured sparsity, and a periodic online testing framework that detects permanent faults with minimal overhead. Together, these techniques enhance the robustness of sparse tensor accelerators for safety-critical machine learning applications. Microarchitecture improvements are also extended to hardware accelerators for Graph Convolutional Networks. More specifically, we propose FusedGCN, a fused triple-matrix systolic architecture that significantly reduces inference latency compared to state-of-the-art accelerators. To enhance reliability, we also introduce GCN-ABFT, a custom fault-tolerance mechanism for GCNs that computes checksum for the whole GCN layer without relying on separate checksums for each matrix multiplication step. Also, we propose a SIMD-based softmax hardware unit that flexibly supports both full-vector and segmented softmax computations. This design enables the unit to compute a complete softmax over an input vector or multiple smaller softmax operations over its sub-vectors. The intermediate results from these sub-vector computations can be efficiently reused to implement various activation functions, such as GELU, sigmoid or tanh. By leveraging this reuse, a single SIMD softmax unit can serve as a unified hardware block for multiple activation functions, reducing hardware redundancy and improving overall resource efficiency. Overall, this work demonstrates that by combining micro-architectural innovation, physical implementation awareness, and application-driven co-design, systolic and tensor array accelerators can be significantly improved in terms of performance, energy efficiency, and reliability. The proposed methods address both the dense and sparse computation regimes of modern ML models, laying the foundation for next-generation hardware that balances speed, energy, and robustness in increasingly demanding application contexts.
Degree
thesis:*- Grantor dc:publisher
- Democritus University of Thrace (DUTH)
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Peltekis, Christodoulos
- Πελτέκης, Χριστόδουλος
Subjects
dc:subject × 25- Συστολικοί πίνακες
- Αραιοί συστολικοί πίνακες
- Δομημένη αραιότητα
- Επιταχυντές υλικού
- Ενεργειακή αποδοτικότητα
- Ανοχή σε σφάλματα
- Συνελικτικά νευρωνικά δίκτυα
- Συνελικτικά νευρωνικά δίκτυα γράφων
- Transformers
- Μηχανική μάθηση
- Systolic arrays
- Sparse tensor arrays
- Structured sparsity
- Hardware accelerators
- Energy efficiency
- Fault-tolerance
- Convolutional neural networks
- Graph convolutional networks
- Machine learning inference
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/60319