Back to search

Georgia Institute of Technology

Energy-Efficient Hardware Acceleration of Transformer-Based Models

Abstract

dc:description.abstract

This research presents a software-hardware co-optimization framework for energy-efficient deployment of transformer models on FPGAs. It introduces novel quantization techniques tailored for both BERT and generative LLMs. For weight quantization, the proposed Hessian-based parameter-wise method assigns optimal bit precision based on sensitivity analysis, while the proposed row-wise quantization enhances hardware efficiency by converting mixed-precision matrices into two uniform-precision blocks. For attention activations, the proposed Weight-Hessian-aware KV cache quantization applies intra-layer mixed-precision using precomputed sensitivities, eliminating runtime overhead. To further improve hardware efficiency, the proposed Query-Key coupled scheme aligns bit precision within each outer product pair, reducing implementation complexity. A concurrent quantization approach jointly optimizes row-wise weight and Query-Key activation precision using multi-precision formats, improving both compression and energy efficiency. These techniques are implemented on a novel multi-precision FPGA accelerator for BERT and GPT-2, capable of handling both power-of-two and non-power-of-two bit-widths. With optimized dataflow, the design minimizes off-chip memory access and significantly outperforms prior solutions in both energy efficiency and inference performance.

Degree

thesis:*
Level thesis:degree_level
Doctoral
Department dc:contributor.department
Electrical and Computer Engineering
Grantor dc:publisher
Georgia Institute of Technology
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Byun, Woohong
Advisor dc:contributor.advisor
  • Mukhopadhyay, Saibal
Committee members dc:contributor.committeemember
  • Yu, Shimeng
  • Sathe, Visvesh
  • Hao, Callie
  • Kim, Hyesoon

Subjects

dc:subject × 16

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/78665

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Byun, Woohong. Energy-Efficient Hardware Acceleration of Transformer-Based Models. Doctoral thesis, Georgia Institute of Technology, 2025. https://hdl.handle.net/1853/78665