At the end of the 2024 ARC Prize competition, two new verified submissions achieved state-of-the-art scores on the ARC-AGI Public Leaderboard (ARC-AGI-Pub). Jeremy Berman scored 53.6% using an evolutionary test-time compute approach with Claude Sonnet 3.5, while a combined MIT and Cornell team reached 47.5% accuracy via test-time training.
Jeremy Berman's method iteratively generates and refines Python transform functions to avoid local maxima, producing up to 500 functions per task. The MIT/Cornell collaboration leverages complementary induction and transduction techniques, improving accuracy by 6x over base fine-tuned models on an 8B-parameter language model.
These results demonstrate substantial progress in testing frontier models against the ARC-AGI benchmark using relaxed compute constraints.