The authors introduce Quantization-Aware Healing (QAH), a pipeline that distills 4-bit students directly from uncompressed models to recover performance lost during structural compression and quantization. Applied to GPT-OSS 120B, this method produces Hypernova-60B, which matches or beats the bfloat16 source on 7 of 9 benchmarks while using roughly 4 times less weight memory.
- QAH distills the student directly from the original uncompressed model rather than re-fitting a compressed checkpoint to hard labels.
- The approach reaches a comparable peak performance about 7 times faster than quantization-aware training (QAT) and remains stable under continued training.
- Hypernova-60B is released as an open-weight model with half the teacher's parameter count.
The authors aim to provide a deployable recipe that avoids multi-week hyper-parameter searches, noting significant quality gaps between distributed-training backends.