RuBench 1.0 introduces Russian-language repository-level agentic coding benchmark

The authors introduce RuBench 1.0, a benchmark of 25 tasks mined from recent fix commits in five live open-source repositories (aiohttp, aiogram, Laravel, NestJS, Fastify). Each task is specified natively in Russian and judged by upstream maintainer regression tests withheld from release.

• The benchmark evaluates deployed product configurations including Claude Code with Opus 4.8, Sonnet 5, and Haiku 4.5, and Codex CLI with GPT-5.5.
• All 25 fix commits postdate the training-data cutoffs of every evaluated model to ensure contamination control.
• The best configuration resolves 78.7% of tasks, with pass@1 reported alongside task-level confidence intervals, paired comparisons, dollar cost, and token usage.
• Auditing revealed that Claude Code + Fable 5 silently substituted the model on 20% of tasks, rerouting fixes to Opus 4.8.

The authors release task statements, metadata, full agent trajectories, and diffs, while withholding grading oracles with a SHA-256 manifest committed at publication time.