University of Patras
Architecture of multiple processing units for accelerating deep neural network inference
Abstract
dc:descriptionDeep neural network (DNN) inference increasingly relies on specialized hardware for high computational efficiency. This thesis presents a scalable architecture for DNN inference acceleration, prototyped on an FPGA, with a focus on general matrix-matrix multiplication (GEMM)-based operations. Also, it develops a hardware emulator to support system-level architectural exploration of analog in-memory computing (AIMC) and its application to DNN inference. We introduce a dynamically configurable accelerator featuring systolic arrays, high-bandwidth memory, and UltraRAMs. In addition, we implement two configurations of the processing unit (PU) architecture, with different compute performance, using the same interfaces and peripheral blocks. By instantiating multiple PUs and employing a heuristic scheduling strategy for weight transfers, the proposed architecture achieves higher throughput efficiency over prior works. To orchestrate multi-PU execution without using a single deployment strategy, the thesis treats coordination as an instruction-programmable layer, allowing the same hardware fabric to be systematically explored across different parallelism and scheduling policies. To this end, we present an instruction-based coordination architecture with multiple PUs to accelerate DNN inference. This architecture enables programmable multi-PU synchronization through instruction controller units coupled with instruction synchronization units, utilizing instruction types organized into load, compute, and store functional groups. A compilation framework is also presented that transforms DNN models into executable instruction programs, enabling flexible partitioning of the DNN graph into topologically contiguous subgraphs mapped to available PUs. Multiple deployment strategies are supported, enabling pipeline parallelism among PUs and batch-level parallelism across different PU subsets, with runtime switching capabilities among them without FPGA reconfiguration. The proposed approach enables design space exploration, supporting dynamic trade-offs between single-batch and multi-batch performance to accelerate DNN inference. Experimental results on ResNet-50 demonstrate compute efficiency of up to 98%, throughput efficiency gains of up to 2.7×, and energy efficiency improvements of up to 2.9× over prior works across different configurations. Beyond DNN inference acceleration, the thesis examines the system-level impact of device-level non-idealities in memristive crossbar accelerators, where architectural decisions must be evaluated under realistic device variability and noise. AIMC is an emerging non-von Neumann computing paradigm in which certain computational tasks are performed in memory by exploiting the physical properties of the non-volatile memory (NVM) devices. Memristive devices, such as phase-change memory (PCM), store information in conductance levels and are well-suited for in-memory computing. In particular, memristive devices, when organized in a crossbar configuration, can be used to perform matrix-vector multiply operations by exploiting Kirchhoff's circuit laws. To explore the feasibility of AIMC cores for applications such as DNN inference and system-level architectural exploration, it is highly desirable to develop an accurate hardware emulator that captures the key physical attributes of memristive devices. Here, we present such an emulator for PCM, implemented on an FPGA, and experimentally validate it using measurements from a PCM prototype chip. Moreover, we present an application of the emulator for neural network inference, in which it consistently captures the conductance evolution of approximately 400,000 PCM devices. Finally, we outline how the PU architecture can be extended to emulate the noise characteristics of NVM devices, to aid next-generation heterogeneous AIMC-based chip designs by enabling system-level accuracy assessment for DNN inference applications.
Degree
thesis:*- Grantor dc:publisher
- University of Patras
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Petropoulos, Anastasios
- Πετρόπουλος, Αναστάσιος
Subjects
dc:subject × 17- Βαθιά νευρωνικά δίκτυα
- Επιταχυντές υλικού
- Συστολικές διατάξεις
- Εξομοιωτές υλικού
- Deep neural network (DNN) inference
- Hardware accelerators
- Field-programmable gate arrays (FPGAs)
- Systolic arrays
- General matrix-matrix multiplication (GEMM)
- In-memory computing
- Hardware emulators
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- eng
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/61092