The turboquant project has released version 0.3.0, which includes a three-line patch correcting a long-standing precision issue in llama.cpp's CUDA code affecting Tesla P100 GPUs.
- The fix extends an existing exemption for sm_61 architecture to include sm_60 (Pascal P100), forcing correct fp16 math instead of the previous buggy path.
- Benchmarking on Qwen3.6-27B showed median KL divergence improved from 0.0023 to 0.000001, with top-token agreement rising from 96.5% to 99.9%.
- Performance benchmarks indicate decode speed increased by approximately 1.4%, while prefill remained identical within noise.
- The patch is verified to have zero effect on other architectures like Volta, Ampere, or Blackwell, which use different kernels.
This update significantly improves the accuracy and value of Tesla P100s for local LLM inference, addressing a bug that previously degraded output quality without providing performance benefits.