Georgia Institute of Technology
Advancing Expressibility and Scalability in Graph Learning
Abstract
dc:description.abstractGraph learning models have become the dominant approach for learning from graph-structured data, yet two fundamental challenges persist: expressiveness and scalability. Advances in model expressiveness often compromise scalability, underscoring the need for holistic approaches that address both objectives simultaneously. This thesis tackles these interrelated challenges through a model co-design perspective, developing methods that jointly optimize for expressiveness and scalability. We present four key contributions. First, MG-GCN, a multi-GPU framework for efficient full-batch Graph Convolutional Network training, establishes foundational strategies for scaling Graph Neural Networks through 1D row-wise partitioning, memory optimization via buffer reuse, and latency hiding through overlapped communication and computation. Second, we introduce the Graph-Enhanced Contextual Operator (GECO), an architectural innovation that integrates neighborhood propagation with global convolutions to capture long-range dependencies in quasilinear time, enabling scalable Graph Transformers without sacrificing quality. Third, we propose Global Neighbor Sampling (GNS) and Global Layer Sampling (GLS), efficient sampling strategies that overcome the quadratic complexity of dense Graph Transformers by distinguishing local from global neighbors, scaling these models to graphs with millions of nodes. Fourth, we introduce Neighbor Aware Skip Connections (NASC), a lightweight mechanism that adaptively balances node and neighborhood information, serving as a versatile plug-in for both GNNs and Graph Transformers with minimal overhead. Together, these contributions demonstrate that achieving scalability and expressiveness in graph learning does not necessitate increasingly complex architectures. Rather, a model co-design approach—integrating system-level optimizations, architectural innovations, sampling strategies, and simplified components—can deliver high performance efficiently.
Degree
thesis:*- Name thesis:degree_name
- Computer Science, PhD
- Department thesis:degree_department
- Computational Science and Engineering
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Sancak, Kaan
- Advisor dc:contributor.advisor
-
- Çatalyürek, Ümit V.
- Committee members dc:contributor.committeemember
-
- Chau, Duen Horng (Polo)
- Kumar, Srijan
- Prakash, B. Aditya
- Yinglong Xia
- ORCID person:identifier.orcid
Rights
- Language dc:language.iso
- English
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/80255