Georgia Institute of Technology
Less is More: Accelerating Vision by Eliminating Redundancy
Abstract
dc:description.abstractThe key to modern machine learning is scale. With more data, bigger models, and more compute, as a community, we've found that the problems once deemed impossible for a computer to solve have rapidly become attainable---many even becoming easy with today's techniques. But as the scale of modern machine learning has ballooned, so too has its cost. Large transformer models, for instance, can require multiple hundreds of GPUs to train effectively and can be similarly unwieldy to deploy. In this dissertation, I aim to reduce those costs. Specifically, this work focuses on Vision Transformers (ViTs), which have been the dominant driving force in scaling machine learning for computer vision. Over the course of this dissertation, I show that these ViTs perform redundant computation, and that by exploiting these redundancies, we can greatly increase the efficiency of these systems, both during training and inference. In Part 1, I show that we can reduce the amount of spatial computation required by these transformers without losing performance. In Part 2, I show that certain architectural components are redundant and can be removed or greatly simplified. In Part 3, I show how we can exploit redundant features within models to speed them up and to circumvent training. Finally, in Part 4, I show that these speed-ups can compound on each other, resulting in a much faster model.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Interactive Computing
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Bolya, Daniel
- Advisor dc:contributor.advisor
-
- Hoffman, Judy
- Committee members dc:contributor.committeemember
-
- Hays, James
- Kira, Zsolt
- Batra, Dhruv
- Feichtenhofer, Christoph
Subjects
dc:subject × 4Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/75233