lab NVIDIA Research · 14h ago · research

Masters distills vision-language models via masking teacher and reinforcing student

from English

Researchers propose Masters, a mask-progressive reinforcement learning framework for distilling large-scale vision-language models into compact versions suitable for edge deployment. The method addresses the instability caused by the size gap between teacher and student models.

Masters masks non-dominant weights of the teacher to reduce complexity, then progressively restores capacity during training to allow smoother learning.
It integrates an offline RL stage using pre-generated responses from masked teachers, avoiding the computational cost of online think-answer paradigms.
The framework employs two complementary rewards: an accuracy reward for response correctness and a distillation reward quantifying transfer ease.

This approach enables students to achieve strong performance by leveraging rich yet efficient guidance without requiring lengthy response generation processes.

Importance 1/3 Trust 3/3 NVIDIA Research Research paper Training methods

Read original

lab NVIDIA Research · 15h ago

ZPPO uses prompts instead of gradients to improve small vision-language model training

Researchers introduce Zone of Proximal Policy Optimization (ZPPO), a method that injects teacher knowledge into prompts rather than policy gradients to address brittleness in knowledge distillation and drift in reinforcement learning. ZPPO constructs Binary Candidate-included Questions (BCQ) and Negative Candidate-included Questions (NCQ) for hard questions, recirculating them via a replay buffer until the student's accuracy improves or they are evicted.

arxiv arXiv cs.LG · 5d ago

U-OPSD enables unsupervised on-policy self-distillation for LLMs

Researchers propose Unsupervised On-Policy Self-Distillation (U-OPSD), a method that achieves post-training improvement for large language models using only the model's own generations without external supervision. The approach samples multiple rollouts to construct a pseudo-solution via majority vote, then distills a teacher distribution into prefixes of the model's longest incorrect completion.

arxiv arXiv cs.CL · 12d ago · 3 views

Memory Decoder at Scale: Scaling parametric long-term memory to 6.9B parameters

Researchers present Memory Decoder at Scale, a system that scales parametric long-term memory models up to 6.9B parameters and pretrains them on 300B tokens. To address the infeasibility of standard Faiss pipelines at this data scale, the authors implement a distributed indexing pipeline with sparse, batch-wise loading of kNN distributions.

arxiv arXiv cs.CL · 14d ago

Relay-OPD reduces training trajectory length by over 50% in on-policy distillation

Researchers introduce Relay On-Policy Distillation (Relay-OPD) to address prefix failure in standard on-policy distillation, where student errors lead to unreliable supervision. The method uses a teacher-student continuation asymmetry as a trigger for the teacher to briefly take over and redirect the trajectory before the student resumes.

media Hugging Face Forums · 18d ago

ThetaScan v0.1 releases open scan-parallel nonlinear memory with 17M LM results

A research preview of ThetaScan v0.1, a fixed-state nonlinear token mixer, has been released as an open-source implementation. The architecture ensures that every token computes its memory write from the current input and shared parameters rather than an evolving fast state, allowing writes to compose through an associative prefix scan.