Abstract
dc:description.abstractReasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding, yet struggle with advanced generalization tasks such as Abstraction and Reasoning Corpus (ARC) puzzles. To address this, we propose a diverse inference approach that aggregates multiple models and methods at test time to enhance the performance. We automatically verify correctness of solutions to ARC puzzles by code. Our approach increases the success rate from 53% to 69.5% without o3 and 91.5% to 93.75% with o3 which exceeds average human correctness which is 73.3 - 77.2% . Our approach succeeds in solving ARC puzzles that state-of-the-art reasoning LLMs and 948 humans could not. It solves 26.5% of ARC puzzles that o3 high compute does not solve and 80% of ARC puzzles that 948 humans could not. We identify the relationship between the number of diverse models and methods and the performance on verifiable problems. We automate the MARC method—previously evaluated manually—through the implementation of an agentic AI system using Rivet, enabling modular, scalable, and autonomous execution of ARC problem-solving pipelines. This work advances progress toward building flexible, generalizable reasoning systems aligned with the goals of Artificial General Intelligence.
Degree
thesis:*- Name thesis:degree_name
- Master of Science
- Level thesis:degree_level
- masters
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Boston University
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Hyun, Seunghwan
- Advisor dc:contributor.advisor
-
- Drori, Iddo
- ORCID dc:identifier.orcid
Subjects
dc:subject × 6Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/53412