The authors introduce GameEngineBench, a benchmark designed to evaluate coding agents on scoped C++ implementation tasks within Unreal Engine 5 projects. Built from nine real-world game repositories, the evaluation set consists of 110 tasks covering gameplay mechanics, multiplayer behavior, AI, animation, UI, and rendering plugins.
- The tasks require models to make native C++ changes that compile and satisfy behavioral tests within executable Unreal Engine projects.
- Across twelve evaluated configurations, the strongest model achieves a pass@1 rate of 55.5%.
- 31 tasks remain unsolved by every configuration tested.
The results demonstrate that frontier coding agents continue to struggle with deeply integrated C++ development for real-time interactive software, highlighting game-engine benchmarks as a valuable complement to existing software engineering evaluations.