A February 2026 analysis using decontaminated benchmarks like SWE-rebench exposes significant performance gaps between leading AI coding models that are masked by traditional static leaderboards. The study highlights how benchmark contamination inflates scores for some models, making them appear competitive with top-tier options when they are not.

  • MiniMax M2.5 scored 80.2% on SWE-bench Verified but dropped to 39.6% on the decontaminated SWE-rebench, while Claude Opus 4.6 maintained a lead at 51.7%.
  • The top 10 models on SWE-rebench are exclusively from Anthropic, OpenAI, and Google, with Chinese models like Kimi K2 and GLM-5 ranking lower.
  • Gemini 3 Pro dominates algorithmic challenges in LiveCodeBench but trails Claude Opus 4.6 in real-world software engineering tasks requiring multi-file edits.
  • SpecWeave defaults to Claude Opus 4.6 for complex implementation due to its superior performance on rigorous decontaminated benchmarks.

The data suggests that the gap between Tier 1 and Tier 3 models is substantial, with better models reducing the need for retries and manual cleanup despite higher per-problem costs.