Benchmark · coding
Codeforces (Elo)
Codeforces (Elo) measures a model's competitive-programming ability by having it solve timed contest problems, reported as a Codeforces Elo rating (roughly 0–4000, where about 2000+ is strong).
Read more
- Example
- A typical item is a timed algorithmic puzzle — for example, read a problem's input and print the correct answer within strict time and memory limits, using techniques like graph search, dynamic programming, or greedy algorithms.
- Scoring
- The result is a single Elo number computed from how many contest problems the model solves and how hard they are, placed on the same rating scale Codeforces uses for human competitors.
- Verification
- Each submitted solution is judged automatically: the code is compiled and run against a hidden set of test cases and only counts as solved if it produces correct output within the time and memory limits.
- Why it matters
- It reflects multi-step algorithmic reasoning and the ability to write correct, efficient code under constraints, and maps a model's coding skill onto a rating scale people already understand.
Worked example
"Codeforces (Elo)" gives a model competitive-programming problems and reports its strength as an Elo rating on the same scale humans get on the site. A representative easy item (Div. 2 A, ~800): "You get t test cases; each gives n and an array a_1…a_n. In one operation you pick i≠j and do a_i−=1, a_j+=1. Print YES if you can make all elements equal, else NO." For the input `2 / 3 / 1 2 3 / 2 / 1 2` the expected output is `YES` then `NO`. The trick is that the operation preserves the total sum S=Σa_i, so equal elements (each S/n) are reachable iff n divides S: [1,2,3] has S=6 divisible by 3 → YES, while [1,2] has S=3 not divisible by 2 → NO. A correct submission reads stdin, prints these answers, and must pass all hidden tests within the time limit; the model's Elo is inferred from the difficulty of the problems it reliably solves.
No verified scores reported yet for this benchmark.