Boston University
Test-time fine-tuning via diverse rewards: a conditional variational reward model approach on the abstraction and reasoning corpus
Abstract
dc:description.abstractThis thesis introduces a framework for test-time adaptation of AI systems that leverages diverse reward signals to refine and optimize performance on challenging tasks, with a focus on the Abstraction and Reasoning Corpus (ARC). At its core, the proposed approach employs a Conditional Variational Reward Model (CVRM) to learn a distribution over reward functions, thereby capturing uncertainty and accommodating conflicting objectives. By integrating CVRM with Large Language Model (LLM)–based program synthesis, the system generates candidate solutions that are iteratively refined through Group Relative Policy Optimization (GRPO) during test time. This dynamic fine-tuning process allows the model to adapt “on the fly,” improving its reasoning and abstraction capabilities with minimal examples and computational overhead. Extensive experiments and ablation studies evaluate the impact of various reward aggregation metrics, LoRA module configurations, and self-correction prompt depths, demonstrating potential robustness and performance on ARC tasks and we also include our top ranking solution for the ARC kaggle competition.
Degree
thesis:*- Name thesis:degree_name
- Master of Science
- Level thesis:degree_level
- masters
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Boston University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Oruganti, Shivacharan
- Advisor dc:contributor.advisor
-
- Ohn-Bar, Eshed
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/53414