Researchers demonstrate that distilling an 8B reasoning teacher (deepseek-r1:8B) into a 0.6B student model (Qwen3-0.6B via QLoRA) allows for high-volume structured extraction with significantly lower latency and cost. The study evaluates this approach on mapping news articles to JSON objects containing summaries and categorical labels, comparing the distilled model against few-shot prompting and constrained decoding baselines.
- The student model processes each article in approximately 0.8 seconds, compared to the teacher's 39 seconds, recovering 58% of the performance gap relative to the base model.
- It outperforms constrained decoding by +16.8 points and few-shot prompting by +4.9 points in summary quality scores from a blinded three-judge panel.
- The reasoning capability of the teacher is critical for writing quality, as distilling from a same-size non-reasoning teacher yields no improvement over the untuned base model.
- While the reasoning lineage improves summary generation, it causes more fabrication on thin-source articles compared to an instruction-based teacher, which maintains better grounding.
The findings suggest that because no single engine wins every field, the optimal approach involves a per-field routing map for on-device enrichment rather than relying on a single model.