A university HPC cluster benchmarked Qwen 3.6 quantizations, finding that while knowledge scores remain stable, lower-precision formats significantly degrade agentic capabilities.
- GPQA Diamond results show minimal variation across different quantization levels.
- Terminal-Bench 2 scores exhibit significant regression in lower-precision quantizations.
- Results are lower than Qwen's official FP8 benchmarks due to shorter timeout settings (10 minutes to 1 hour vs. 3 hours).
- Multiple runs reveal considerable score variation, where a poor high-precision run can underperform a good low-precision one.
The findings help users understand the trade-offs of quantization for agentic tasks versus knowledge retention.