Databricks published a benchmark of coding agents on its multi-million-line codebase, reporting that its internal pi-coding-agent is up to 2x cheaper than Claude Code or Codex while achieving higher pass rates. The study also found that GLM 5.2 performs on par with Opus 4.8 high and exceeds GPT 5.5 high and xhigh.
- pi-coding-agent uses a bash-for-everything approach with minimum tools, resulting in lower costs and better pass rates across the board compared to Claude Code and Codex.
- GLM 5.2 is ranked above GPT 5.5 high and xhigh, and is considered on par with Opus 4.8 high for most coding tasks.
- The author notes that Claude Code's prefix includes built-in tools like Playwright which may benefit visual tasks, while GLM lacks native image input support.
The benchmark provides an independent evaluation from Databricks, a lab with significant LLM training experience, offering insights into the cost-performance trade-offs of different coding agents.