Benchmark · math
MATH-500
MATH-500 is a 500-problem subset of the MATH competition-mathematics dataset, used to test a model's ability to solve challenging math problems. The score is the percentage of problems whose final answer the model gets correct.
Read more
- Example
- A typical item is a competition-style math problem — for instance an algebra, geometry, or number-theory question that asks for a single final answer (like a number or a closed-form expression) after several steps of reasoning.
- Scoring
- The score is the share of the 500 problems the model answers correctly, i.e. accuracy = correct final answers / 500, reported as a percentage.
- Verification
- A problem counts as solved when the model's final answer matches the reference answer exactly (an automatic equivalence check of the final answer, not of the intermediate steps).
- Why it matters
- It is a compact, widely used yardstick for mathematical reasoning that runs faster than the full MATH set, so it is a common quick check when comparing reasoning models.
Worked example
Task
For certain values of $k$, the quadratic $x^2 - kx + 16 = 0$ has only positive integer roots. Find the sum of all distinct possible values of $k$.
Solution
Roots $r,s$: $rs=16$, $k=r+s$. Pairs $(1,16),(2,8),(4,4)\Rightarrow k\in\{17,10,8\}$. Distinct sum $=17+10+8=\boxed{35}$.
Walkthrough
The roots $r,s$ are positive integers with $rs=16$ and $k=r+s$; the factor pairs $(1,16),(2,8),(4,4)$ give $k=17,10,8$, so the distinct sum is $35$. MATH-500 grades by exact match of the normalized final \boxed{} answer, so only $35$ counts.