Back to search

Boston University

Solving the ARC-AGI challenge at a human level

Abstract

dc:description.abstract

Reasoning LLMs such as OpenAI o1, o3 and DeepSeek R1 have made significant progress in mathematics and coding, yet struggle with advanced generalization tasks such as Abstraction and Reasoning Corpus (ARC) puzzles. To address this, we propose a diverse inference approach that aggregates multiple models and methods at test time to enhance the performance. We automatically verify correctness of solutions to ARC puzzles by code. Our approach increases the success rate from 53% to 69.5% without o3 and 91.5% to 93.75% with o3 which exceeds average human correctness which is 73.3 - 77.2% . Our approach succeeds in solving ARC puzzles that state-of-the-art reasoning LLMs and 948 humans could not. It solves 26.5% of ARC puzzles that o3 high compute does not solve and 80% of ARC puzzles that 948 humans could not. We identify the relationship between the number of diverse models and methods and the performance on verifiable problems. We automate the MARC method—previously evaluated manually—through the implementation of an agentic AI system using Rivet, enabling modular, scalable, and autonomous execution of ARC problem-solving pipelines. This work advances progress toward building flexible, generalizable reasoning systems aligned with the goals of Artificial General Intelligence.

Degree

thesis:*
Name thesis:degree_name
Master of Science
Level thesis:degree_level
masters
Discipline thesis:degree_discipline
Computer Science
Grantor
Boston University
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hyun, Seunghwan
Advisor dc:contributor.advisor
  • Drori, Iddo
ORCID dc:identifier.orcid

Subjects

dc:subject × 6

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/53412

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Hyun, Seunghwan. Solving the ARC-AGI challenge at a human level. masters thesis, Boston University, 2025. https://hdl.handle.net/2144/53412