LFM2.5 Q4_0 checkpoints use Quantization-Aware Distillation to recover 97% of BF16 accuracy

The LFM2.5 team has released updated 4-bit GGUF checkpoints for the LFM2.5-230M, LFM2.5-350M, LFM2.5-1.2B-Instruct, and LFM2.5-2.6B models, trained using Quantization-Aware Distillation (QAD). This technique distills a high-precision teacher model into a quantized student to recover accuracy lost during standard quantization.

• QAD recovers approximately 97% of the performance gap between BF16 and standard Q4_0 post-training quantization across all four model sizes.
• Specific recovery rates are 70.6% for LFM2.5-230M, 73.4% for LFM2.5-350M, 65.5% for LFM2.5-1.2B, and 48.4% for LFM2.5-2.6B.
• Benchmarks covering reasoning, instruction-following, tool use, and agentic capabilities were conducted on GPQA Diamond, MMLU-Pro, IFEval, IFBench, Multi-IF, BFCLv4, GSM8K, and AIME25.
• The checkpoints maintain the high throughput of native Q4_0 GGUFs while matching or exceeding the quality of Q5_K_M and Q4_K_M formats on tested hardware backends.

These updates allow edge devices to retain low memory footprints and high inference speeds without significant accuracy degradation compared to full-precision models.