Back to search

Georgia Institute of Technology

The Algorithm-Hardware Specialization Spectrum of Multi-Stage Intelligence Pipelines: From Dedicated Accelerators to Heterogeneous Systems

Abstract

dc:description.abstract

This dissertation develops a hardware-centric specialization framework for multi-stage intelligence pipelines, using 3D intelligence as the primary study domain. Pipelines in this domain span perception and reconstruction, rendering, and high-level reasoning; their stages differ fundamentally in computational regularity, memory behavior, and control-flow dynamics, and therefore interact with current hardware to very different degrees. We characterize this heterogeneity through algorithmic entropy, a hardware-oriented measure of execution unpredictability that decomposes into two orthogonal axes: intra-operator entropy (X, datapath irregularity) and inter-operator entropy (Y, scheduling unpredictability), quantified via GPU profiling across nine representative workloads. The resulting two-dimensional space partitions workloads into quadrants, each mapping to a substrate along a hierarchical specialization spectrum: Q1 (low X, low Y) → dedicated ASIC; Q2 (high X, low–moderate Y) → enhanced fixed-function GPU; Q3 (low X, high Y) → heterogeneous GPU-PIM; Q4 (high on both axes) → commodity GPU/CPU baseline. Three systems instantiate this principle, one per populated quadrant: Fusion-3D (Q1): a dedicated 3D-reconstruction ASIC with hierarchical spatial tiling and a unified on-chip pipeline, extended to multi-chip for large scenes. Achieves 2.5× / 6× throughput over prior accelerators in reconstruction and inference, validated on a silicon prototype. GauRast (Q2): a 3D Gaussian Splatting rasterizer that extends the existing GPU fixed-function rasterizer with lightweight neural-rendering operations. Achieves 6× / 4× end-to-end speedup on original / optimized 3DGS pipelines at ≤0.2% SoC area overhead. ORCHES (Q3): a GPU-PIM heterogeneous system for test-time-compute (TTC) LLM/VLM reasoning, combining adaptive workload assignment, branch-prediction-guided pipelining, and fragmentation-aware memory structuring. Achieves 4.16× / 3.10× end-to-end speedup on text / vision reasoning over SOTA GPU baselines. Across the three quadrants, profile-guided substrate selection delivers substantial speedups for 3D intelligence workloads while specializing only the axis where existing hardware fails. Beyond 3D intelligence, we anticipate the same (X, Y) methodology serving as a starting point for substrate selection in other multi-stage, heterogeneous-workload domains.

Degree

thesis:*
Name thesis:degree_name
Computer Science, PhD
Department thesis:degree_department
Computer Science
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Li, Sixu
Advisors dc:contributor.advisor
  • Lin, Yingyan (Celine)
  • Hester, Josiah
Committee members dc:contributor.committeemember
  • Krishna, Tushar
  • Tambe, Thierry
  • Kim, Hyesoon
ORCID person:identifier.orcid

Rights

Language dc:language.iso
English

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82155

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Li, Sixu. The Algorithm-Hardware Specialization Spectrum of Multi-Stage Intelligence Pipelines: From Dedicated Accelerators to Heterogeneous Systems. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82155