Back to search

University of Patras

Architecture of multiple processing units for accelerating deep neural network inference

Abstract

dc:description

Deep neural network (DNN) inference increasingly relies on specialized hardware for high computational efficiency. This thesis presents a scalable architecture for DNN inference acceleration, prototyped on an FPGA, with a focus on general matrix-matrix multiplication (GEMM)-based operations. Also, it develops a hardware emulator to support system-level architectural exploration of analog in-memory computing (AIMC) and its application to DNN inference. We introduce a dynamically configurable accelerator featuring systolic arrays, high-bandwidth memory, and UltraRAMs. In addition, we implement two configurations of the processing unit (PU) architecture, with different compute performance, using the same interfaces and peripheral blocks. By instantiating multiple PUs and employing a heuristic scheduling strategy for weight transfers, the proposed architecture achieves higher throughput efficiency over prior works. To orchestrate multi-PU execution without using a single deployment strategy, the thesis treats coordination as an instruction-programmable layer, allowing the same hardware fabric to be systematically explored across different parallelism and scheduling policies. To this end, we present an instruction-based coordination architecture with multiple PUs to accelerate DNN inference. This architecture enables programmable multi-PU synchronization through instruction controller units coupled with instruction synchronization units, utilizing instruction types organized into load, compute, and store functional groups. A compilation framework is also presented that transforms DNN models into executable instruction programs, enabling flexible partitioning of the DNN graph into topologically contiguous subgraphs mapped to available PUs. Multiple deployment strategies are supported, enabling pipeline parallelism among PUs and batch-level parallelism across different PU subsets, with runtime switching capabilities among them without FPGA reconfiguration. The proposed approach enables design space exploration, supporting dynamic trade-offs between single-batch and multi-batch performance to accelerate DNN inference. Experimental results on ResNet-50 demonstrate compute efficiency of up to 98%, throughput efficiency gains of up to 2.7×, and energy efficiency improvements of up to 2.9× over prior works across different configurations. Beyond DNN inference acceleration, the thesis examines the system-level impact of device-level non-idealities in memristive crossbar accelerators, where architectural decisions must be evaluated under realistic device variability and noise. AIMC is an emerging non-von Neumann computing paradigm in which certain computational tasks are performed in memory by exploiting the physical properties of the non-volatile memory (NVM) devices. Memristive devices, such as phase-change memory (PCM), store information in conductance levels and are well-suited for in-memory computing. In particular, memristive devices, when organized in a crossbar configuration, can be used to perform matrix-vector multiply operations by exploiting Kirchhoff's circuit laws. To explore the feasibility of AIMC cores for applications such as DNN inference and system-level architectural exploration, it is highly desirable to develop an accurate hardware emulator that captures the key physical attributes of memristive devices. Here, we present such an emulator for PCM, implemented on an FPGA, and experimentally validate it using measurements from a PCM prototype chip. Moreover, we present an application of the emulator for neural network inference, in which it consistently captures the conductance evolution of approximately 400,000 PCM devices. Finally, we outline how the PU architecture can be extended to emulate the noise characteristics of NVM devices, to aid next-generation heterogeneous AIMC-based chip designs by enabling system-level accuracy assessment for DNN inference applications.

Degree

thesis:*
Grantor dc:publisher
University of Patras
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Petropoulos, Anastasios
  • Πετρόπουλος, Αναστάσιος

Subjects

dc:subject × 17

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/61092

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Petropoulos, Anastasios; Πετρόπουλος, Αναστάσιος. Architecture of multiple processing units for accelerating deep neural network inference. University of Patras, 2026. http://hdl.handle.net/10442/hedi/61092