A study demonstrates that section-aware compression of self-distilled reasoning traces allows Qwen3.5-4B and Gemma-4-12b models to match or exceed original performance using significantly fewer tokens. By preserving computation and verification spans while compressing narration, the method avoids the greedy decoding failures associated with flat compression.
- Flat compression causes greedy decoding to fail, looping on 93% of GSM8K problems at temperature 0.
- Section-aware compression improves accuracy by +.15 on GSM8K compared to uncompressed SFT control while using ~1.7x fewer tokens.
- Training bare without a system prompt and serving with it yields the best results, as the prompt acts as an efficiency trigger.
- Compression behavior can be toggled via an identity prompt, allowing the model to switch between concise reasoning and decompressed output.
- A stronger teacher model produced worse results than uncompressed SFT, indicating traces must stay close to the student's distribution.
- The approach transfers effectively to larger models; Gemma-4-12b achieved .86 accuracy on GSM8K at 1,679 tokens versus .57 at 3,753 for the original.
The research highlights that preserving computation spans anchors termination and improves efficiency, with full code, models, and datasets made available.