Cornell University
Improving the ML Tradeoff Curve: Efficiencies in Computation, Memory, and Tuning
Abstract
dc:description.abstractA fundamental idea in machine learning (ML) has been the scaling of resources to increase model capability. More training iterations, hyperparameter tuning, model parameters, or data are generally beneficial. For a computational or memory resource, and a capability metric, there is a a Pareto frontier. Navigating this tradeoff curve, i.e. scaling resources to scale model capabilities, has powered much of the advancements in the modern ML age of the past 15 years. However, there is another way: the tradeoff curve can be improved. Scaling resources is not always viable, be it on edge devices or in data centers. My PhD has focused on this approach. Part I presents my work in post training compression. Given an already trained model, reduce the memory and/or compute footprint with minimal degradation. I propose methods in pruning and quantization. This problem formulation has become especially compelling with the prohibitive cost of creating LLMs. Part II presents my work in reducing hyperparameter tuning load. More tuning generally improves a model, but can be non-trivial. I use meta-learning to estimate optimal hyperparameters, and develop a more hyperparameter efficient statistical inference method. Part III goes beyond this tradeoff framework, which assumes a fixed objective. Improvements in recommender safety require reformulating the problem statement.
Degree
thesis:*- Name thesis:degree_name
- Ph. D., Computer Science
- Level thesis:degree_level
- Doctor of Philosophy
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Cornell University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Chee, Jerry
- Chair dc:contributor.chair
-
- De Sa, Christopher
- Committee members dc:contributor.committeemember
-
- Damle, Anil
- Bunea, Florentina
- Joachims, Thorsten
Subjects
dc:subject × 6Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Bibid
- 16938194
- Dc Identifier Other
-
ProQuest Submission ID: 14899
ProQuest Publication ID: 31997505