Back to search

Rochester Institute of Technology

A Comparative Study of Inference-Time Scaling Strategies for Large Language Models

Abstract

dc:description.abstract

Large language models (LLMs) have demonstrated strong performance on a range of reasoning tasks, however, their reliability often depends not only on model size or training data, but also on inference-time strategies. However, existing inference-time methods are typically evaluated in isolation and under differing experimental assumptions, making it difficult to draw systematic conclusions about their relative effectiveness. This thesis proposes a controlled empirical study of inference-time scaling strategies for large language models under fixed inference-time compute budgets. The findings reveal that no single strategy dominates uniformly. PRM guided selection with the IBM Granite verifier achieves the highest absolute accuracy across arithmetic and compositional tasks. Multi-agent debate with heterogeneous critic-solver pairs achieves the highest overall accuracy on object counting, surpassing PRM guidance. Majority voting provides reliable, compute-efficient improvements across most settings when answer diversity is sufficient, and while beam search helps, it does not usually provide much benefit with width growth. We also show that verifier choice and aggregation method (notably, last-step aggregation) substantially affect PRM performance, and that model heterogeneity in debate consistently outperforms self-debate.

Degree

thesis:*
Name thesis:degree_name
Computer Science (MS)
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Computer Science, Department of

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Owolabi, Oluwamayowa
Contributors dc:contributor
  • Christopher Homan

Subjects

dc:subject × 3

Identifiers

dc:identifier.*
Repository record dc:identifier
https://repository.rit.edu/theses/12550

Chain of custody

source
Harvested from
Rochester Institute of Technology
Base URL
repository.rit.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Owolabi, Oluwamayowa. A Comparative Study of Inference-Time Scaling Strategies for Large Language Models. Thesis thesis, https://repository.rit.edu/theses/12550