Researchers introduce SWE-rebench V2, a language-agnostic automated pipeline designed to harvest executable real-world software engineering tasks and construct reinforcement learning training environments at scale. The system synthesizes repository-specific installation and test procedures via an interactive setup agent and filters unsound instances using an ensemble of LLM judges.
- Constructs a dataset of 32,079 tasks spanning 20 languages and 3,617 repositories with pre-built images for reproducible execution.
- Releases 120,000+ additional tasks with installation instructions, fail-to-pass tests, and rich metadata derived from pull request descriptions.
- Provides instance-level diagnostic metadata to flag common confounders such as overly restrictive tests and underspecified descriptions.
- Validates collected instances through a diagnostic study covering five programming languages across seven popular models.
The release of the datasets, collection code, and associated artifacts enables large-scale training of SWE agents across diverse languages and repositories.