A comparison of GLM-5.3 and its distilled variant, GLM-5.3 Flash, on the DeepSWE benchmark reveals that distillation preserves core coding capability while significantly reducing rollout costs. The full model achieves a 69.0% pass@1 rate at $3.99 per task, whereas the Flash variant scores 63.4% at just $0.24, representing a 17x price reduction.
- The quality gap narrows from 5.6 points at pass@1 to 2.6 points at pass@4 (87.6% vs 85.0%), indicating the loss is primarily reliability rather than capability.
- GLM-5.3 Flash costs $0.24 per rollout compared to $3.99 for the full model, yielding 264 solved tasks per $100 versus 17.
- The Flash variant loses the ability to convert extra effort into success on flaky tasks, with passing runs taking more steps only 46% of the time compared to 61% for the full model.
- Distillation reshaped the performance profile, gaining ground in concurrency, Python, and data modeling while ceding JavaScript and query-heavy work.
- A notable regression is reduced caution, with the Flash breaking baseline tests in 6.9% of rollouts versus 4.4% for the full model.
Running GLM-5.3 Flash first and escalating to the full model only upon rejection solves 80.9% of DeepSWE tasks at $1.70 each, offering a cost-effective strategy for throughput-bound workloads.