A study of 2,848 provider-billed Claude Code runs reveals that reducing retrieved context or tool output does not reliably lower the actual billed cost of coding agents. The research challenges the common evaluation metric of text removal by analyzing a campaign of 2,908 executions across three models and seven repositories.
- Prompt-cache traffic dominated costs, accounting for approximately 87% of reconstructed costs and 80% of the actual bill.
- An arm that removed 38% of estimated raw tool-output tokens actually incurred a 6.8% higher paired cost compared to the baseline.
- Compression was found to harm task completion by removing action-critical evidence, reducing patch application success from 27/40 to 15/40 in SWE-bench-derived Go tasks.
The authors propose using success-adjusted billed cost as a more accurate standard for evaluating efficiency than token reduction alone.