The authors introduce Quantization-Aware Healing (QAH), a method to recover reasoning and coding capabilities in structurally compressed, 4-bit large language models. Unlike standard quantization-aware training which re-fits the compressed model, QAH distills the 4-bit student directly from the original uncompressed model.

  • Applied to GPT-OSS 120B reduced to 60B with MXFP4 quantization, the resulting Hypernova-60B matches or beats its bfloat16 source on 7 of 9 benchmarks.
  • The approach uses roughly 4 times less weight memory and half the teacher's parameter count.
  • QAH reaches a comparable peak performance to quantization-aware training about 7 times faster and remains stable under continued training without hand-tuned early stopping.

The method provides a practical recipe for deploying compressed models without requiring multi-week hyper-parameter searches.