Researchers propose LP-SFT, a Local-Preserving Supervised Fine-Tuning objective designed to protect the inherent multimodal entropy structure of pretrained language models. The method constructs an adaptive support of alternative tokens and applies a locally normalized preservation loss to maintain relative structure while standard cross-entropy optimizes the supervised token.
- Analysis reveals that pretrained models exhibit a regular multimodal entropy structure with peaks corresponding to plausible alternatives.
- LP-SFT explicitly protects this distributional knowledge during adaptation, preventing distortion of local preference structures.
- Experiments show LP-SFT improves overall performance over vanilla SFT and recent baselines in mixed-domain and single-domain settings.
- The approach achieves the best balance between pass@1 accuracy and pass@k performance by mitigating capability degradation.
This method helps maintain sampling-accessible diversity, addressing the common trade-off where fine-tuning improves target-domain behavior at the cost of degrading pre-existing capabilities.