The developers of TrueForge, an open-source model-neutral agent harness, benchmarked it against Claude Managed Agents using the DevRev Enterprise-Bench. The comparison revealed that TrueForge can match the solve rate of managed platforms while significantly reducing resource consumption.
- TrueForge + Opus 4.8 solved 11/14 tasks with 63% fewer tokens and 30% lower cost per run compared to Claude Managed Agents + Opus 4.8.
- The efficiency gain stems from reduced context carrying, compaction, and fewer tool calls (averaging 19 vs 32).
- TrueForge + GLM-5.2 achieved a slightly higher solve rate of 11.7/14 tasks at roughly 75% lower cost than the Claude baseline.
- The MIT-licensed harness supports OpenAI-compatible endpoints, subagents, and persistent sessions but lacks first-class tracing tooling.
The authors highlight that keeping the runtime model-neutral allows users to independently optimize for token efficiency and price-performance by selecting different models.