Researchers propose Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric framework that leverages internal neuron activations to guide training-data selection and teacher context construction for large language models. This approach enables annotation-free self-distillation by training the model via on-policy distillation from the teacher distribution, eliminating the need for ground-truth labels.
The method addresses limitations in existing annotation-free self-evolution techniques, such as out-of-domain performance degradation in SFT- and GRPO-based variants and calibration error inflation in reward-based on-policy RL. By using neuron activations to guide the process, Neuron-OPSD aims to improve in-domain task performance while preserving cross-domain generalization.
The framework is designed for settings where online interaction or external supervision is costly or infeasible, offering a conceptually distinct alternative to offline RL approaches that rely on logged, reward-labeled trajectories.