The article argues that model routing in agentic systems is a systems optimization problem rather than a simple classification task, driven by three key complexities: actual costs depend on caching behavior rather than just sticker price, task difficulty is often invisible at routing time, and latency is dominated by infrastructure state.
- On the AppWorld Test Challenge with a CodeAct agent, Claude Sonnet cost $79 total while GPT-4.1 cost $155 due to Sonnet's lower cache-read pricing benefiting from high hit rates.
- A standard difficulty-based router landed at similar accuracy but higher cost compared to an optimization-based approach that explores the full tradeoff space.
- The authors' lightweight router (6 ms and 2 kB memory per task) achieved a configuration with 84% accuracy, $93 cost, and 83s latency, representing a 21% cost reduction and 9% latency reduction compared to running Opus alone.
The authors conclude that effective routing finds the best operating point for the entire system by balancing cost, quality, latency, compliance, and reliability simultaneously.