Back to search

Georgia Institute of Technology

The Hidden Structure of Deep Models: Discovering and Exploiting Sparse Subnetworks

Abstract

dc:description.abstract

Deep learning's growing scale makes computational, memory, and communication efficiency increasingly important, and sparsity offers a principled way to improve all three. This dissertation treats sparsity as an integral component of learning in three settings: controllable sparse projection, sparse reinforcement learning, and communication-efficient federated learning. First, Group Sparse Projection (GSP) directly controls the average Hoyer sparsity of grouped vectors through one interpretable parameter while allowing individual sparsity levels to adapt. The dissertation establishes the projection's theoretical properties and algorithm, then demonstrates it in sparse nonnegative matrix factorization and neural network pruning. Second, Data Adaptive Pathway Discovery (DAPD) learns sparse reinforcement-learning pathways under changing data distributions. It adapts masks during a warm-up phase and freezes them for stable optimization. Across single-task, multi-task, online, and offline settings, DAPD preserves strong policy performance with a small fraction of a shared network's parameters. Third, Salient Sparse Federated Learning (SSFL) addresses communication-constrained, non-IID federated learning by aggregating clients' private-data saliency scores into a shared mask before training. Experiments across CIFAR-10, CIFAR-100, Tiny-ImageNet, and deeper ResNets show competitive accuracy with less communication. A geographically distributed deployment yields wall-clock savings, and one-shot mask rediscovery adapts to out-of-distribution classes. Together, these methods show that matching sparsity discovery to the learning problem can reduce parameter, computation, and communication costs while preserving competitive performance.

Degree

thesis:*
Name thesis:degree_name
Electrical and Computer Engineering, PhD
Department thesis:degree_department
Electrical and Computer Engineering
Grantor
Georgia Institute of Technology
Year dc:date.issued
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Ohib, Riyasat
Advisor dc:contributor.advisor
  • Calhoun, Vince D.
Committee members dc:contributor.committeemember
  • Plis, Sergey
  • Anderson, David
  • Xiong, Li
  • Malek, Mani
  • Essa, Irfan
ORCID person:identifier.orcid

Rights

Language dc:language.iso
English

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/1853/82225

Chain of custody

source
Harvested from
Georgia Tech
Base URL
repository.gatech.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Ohib, Riyasat. The Hidden Structure of Deep Models: Discovering and Exploiting Sparse Subnetworks. Georgia Institute of Technology, 2026. https://hdl.handle.net/1853/82225