The paper introduces Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric framework that leverages internal neuron activations to guide training-data selection and teacher context construction for large language models. This approach enables annotation-free self-distillation without requiring ground-truth labels or real-world interaction feedback.
- Addresses drawbacks of existing methods where SFT- and GRPO-based variants suffer out-of-domain performance degradation and reward-based on-policy RL inflates calibration error.
- Uses internal neuron activations to guide both training-data selection and teacher context construction.
- Trains the model via on-policy distillation from the teacher distribution, requiring no ground-truth labels at any stage.
Neuron-OPSD improves in-domain task performance while preserving cross-domain generalization and mitigating calibration collapse over prior annotation-free baselines. This framework is particularly relevant to settings where online interaction or external supervision is costly or infeasible.