arxiv arXiv cs.LG · just now · research

U-OPSD enables unsupervised on-policy self-distillation for LLMs

from English

Researchers propose Unsupervised On-Policy Self-Distillation (U-OPSD), a method that achieves post-training improvement for large language models using only the model's own generations without external supervision. The approach samples multiple rollouts to construct a pseudo-solution via majority vote, then distills a teacher distribution into prefixes of the model's longest incorrect completion.

U-OPSD improves base models by 8.5% and 10.7% on Qwen3 non-thinking mode at 4B and 8B scales across benchmarks like AIME24, MATH500, and AMC23.
It outperforms supervised methods such as OPSD by an average of 3.2% and 2.3% in non-thinking mode.
In thinking mode, U-OPSD matches or surpasses OPSD and GRPO, outperforming the latter by up to 1.1% at the 8B scale.

This method demonstrates that genuine self-distillation is possible through internal consistency, offering a competitive alternative to supervised training signals.

Importance 2/3 arXiv cs.LG Research paper Training methods

Read original

arxiv arXiv cs.CL · 7d ago · 2 views

Memory Decoder at Scale: Scaling parametric long-term memory to 6.9B parameters

Researchers present Memory Decoder at Scale, a system that scales parametric long-term memory models up to 6.9B parameters and pretrains them on 300B tokens. To address the infeasibility of standard Faiss pipelines at this data scale, the authors implement a distributed indexing pipeline with sparse, batch-wise loading of kNN distributions.

arxiv arXiv cs.CL · 9d ago

Relay-OPD reduces training trajectory length by over 50% in on-policy distillation

Researchers introduce Relay On-Policy Distillation (Relay-OPD) to address prefix failure in standard on-policy distillation, where student errors lead to unreliable supervision. The method uses a teacher-student continuation asymmetry as a trigger for the teacher to briefly take over and redirect the trajectory before the student resumes.

media Hugging Face Forums · 13d ago

ThetaScan v0.1 releases open scan-parallel nonlinear memory with 17M LM results

A research preview of ThetaScan v0.1, a fixed-state nonlinear token mixer, has been released as an open-source implementation. The architecture ensures that every token computes its memory write from the current input and shared parameters rather than an evolving fast state, allowing writes to compose through an associative prefix scan.

media r/LocalLLaMA · 15d ago · 1 view

SLAI T-Rex achieves 34.22% MFU and 71.81% Pass@1 for DeepSeek-V4 on Ascend

SLAI introduces T-Rex, a full-parameter post-training framework for the DeepSeek-V4 model family on Ascend SuperPOD infrastructure. The system addresses memory pressure and communication overhead in trillion-parameter MoE models through hierarchical optimization.

arxiv arXiv cs.LG · 22d ago · 1 view

Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum

The authors propose using the Lyapunov characteristic exponent (LCE) as a dense reward signal for stabilizing an inverted pendulum with vertical motion. This approach enables reinforcement learning agents to discover complex stabilization behaviors beyond standard oscillatory solutions.