Rochester Institute of Technology
A Comparative Study of Inference-Time Scaling Strategies for Large Language Models
Abstract
dc:description.abstractLarge language models (LLMs) have demonstrated strong performance on a range of reasoning tasks, however, their reliability often depends not only on model size or training data, but also on inference-time strategies. However, existing inference-time methods are typically evaluated in isolation and under differing experimental assumptions, making it difficult to draw systematic conclusions about their relative effectiveness. This thesis proposes a controlled empirical study of inference-time scaling strategies for large language models under fixed inference-time compute budgets. The findings reveal that no single strategy dominates uniformly. PRM guided selection with the IBM Granite verifier achieves the highest absolute accuracy across arithmetic and compositional tasks. Multi-agent debate with heterogeneous critic-solver pairs achieves the highest overall accuracy on object counting, surpassing PRM guidance. Majority voting provides reliable, compute-efficient improvements across most settings when answer diversity is sufficient, and while beam search helps, it does not usually provide much benefit with width growth. We also show that verifier choice and aggregation method (notably, last-step aggregation) substantially affect PRM performance, and that model heterogeneity in debate consistently outperforms self-debate.
Degree
thesis:*- Name thesis:degree_name
- Computer Science (MS)
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Computer Science, Department of
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Owolabi, Oluwamayowa
- Contributors dc:contributor
-
- Christopher Homan
Subjects
dc:subject × 3Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.rit.edu/theses/12550