Benchmark · math
AIME 2025
AIME 2025 tests a model's advanced mathematical reasoning using the 15 problems from the 2025 American Invitational Mathematics Examination, a hard high-school competition. Each answer is an integer from 0 to 999, and a model's score is the percentage of problems solved correctly (often reported as pass@1 or avg@k).
Read more
- Example
- A typical item is a challenging competition problem — for instance, a number-theory or combinatorics question whose solution is a single integer between 0 and 999, such as finding the number of ordered pairs that satisfy a given set of algebraic conditions.
- Scoring
- Each of the 15 problems is graded by exact match on its integer answer, and the score is the fraction (percentage) solved correctly; because answers vary from run to run, results are usually averaged over many samples (avg@k) or reported as pass@1.
- Verification
- Verification is automatic and objective: the model's final integer is compared to the official answer key with no partial credit and no human judging, so a solution is accepted only on an exact numeric match.
- Why it matters
- AIME problems demand multi-step, creative reasoning rather than recall, so the benchmark is a widely watched measure of frontier LLM math ability and a common headline number when new reasoning models are released.
Worked example
Task
Representative AIME-style problem (answer is an integer from 0 to 999): Find the sum of all positive integers n for which √(n² + 85n + 2017) is an integer.
Solution
Complete the square: (2m)² − (2n+85)² = 843 = 3·281. Factor the difference of squares: (2m−2n−85)(2m+2n+85) = 843, giving n = 168 or n = 27. Sum = 168 + 27 = 195.
Walkthrough
Completing the square turns the condition into a difference of squares (2m)² − (2n+85)² = 843 = 3·281, whose only positive factorizations give n = 168 and n = 27, summing to 195. AIME is graded by exact match of the single integer answer (0–999), with no partial credit.