Anthropic has released Claude 3.7 Sonnet, a model designed for real-world business and developer use cases rather than just benchmark optimization. The release introduces the ability to dynamically switch between standard and advanced reasoning modes, allowing users to control the number of tokens allocated to "thinking time."
- Pricing is set at $3/M input tokens and $15/M output tokens, positioning it between OpenAI's o1 and DeepSeek R1.
- The model features a hybrid architecture offering ~200 ms latency in standard mode and up to 15 seconds in extended thinking mode.
- Benchmarks show strong performance in graduate-level reasoning (GPQA Diamond: 78.2% / 84.8%) and coding (SWE-bench: 62.3% / 70.3%), though it struggles with high school math competition scores.
- Independent evaluations indicate that none of the tested models, including Claude 3.7 Sonnet, are reliable for complex math problems, with all scoring around coin-toss levels on SAT-style questions.
The configurable latency and token budget control provide developers with flexibility to balance speed and accuracy without needing separate models for different tasks.