arxiv arXiv cs.CL · 1h ago · research

MiLMMT-46-v1.0 improves multilingual translation via reference-free post-training

from English

Researchers have released MiLMMT-46-v1.0, an open large language model for multilingual machine translation that utilizes reference-free post-training. Starting from the supervised-finetuned MiLMMT-46-v0.1, the team applied Group Relative Policy Optimization (GRPO) with a reward based on two reference-free quality estimation models gated by language identification.

The final model is obtained by linearly interpolating the supervised fine-tuning and reinforcement learning checkpoints.
Across 46 languages, MiLMMT-46-v1.0 consistently improves translation quality over its SFT counterpart.
It outperforms strong open baselines including Seed-X, HY-MT2, and TranslateGemma.
The model achieves leading reference-free scores against proprietary systems like Google Translate, Gemini 3 Pro, and GPT-5.
On-policy distillation was investigated but did not surpass the quality frontier achieved by RL with checkpoint interpolation.

The authors release the models and code to facilitate future research in this area.

Importance 2/3 arXiv cs.CL 01.AI Tencent Research paper Training methods

Related articles

lab NVIDIA Research · 17h ago · 2 views

Masters distills vision-language models via masking teacher and reinforcing student

Researchers propose Masters, a mask-progressive reinforcement learning framework for distilling large-scale vision-language models into compact versions suitable for edge deployment. The method addresses the instability caused by the size gap between teacher and student models.

lab NVIDIA Research · 17h ago

ZPPO uses prompts instead of gradients to improve small vision-language model training

Researchers introduce Zone of Proximal Policy Optimization (ZPPO), a method that injects teacher knowledge into prompts rather than policy gradients to address brittleness in knowledge distillation and drift in reinforcement learning. ZPPO constructs Binary Candidate-included Questions (BCQ) and Negative Candidate-included Questions (NCQ) for hard questions, recirculating them via a replay buffer until the student's accuracy improves or they are evicted.

arxiv arXiv cs.LG · 5d ago

U-OPSD enables unsupervised on-policy self-distillation for LLMs

Researchers propose Unsupervised On-Policy Self-Distillation (U-OPSD), a method that achieves post-training improvement for large language models using only the model's own generations without external supervision. The approach samples multiple rollouts to construct a pseudo-solution via majority vote, then distills a teacher distribution into prefixes of the model's longest incorrect completion.

arxiv arXiv cs.CL · 12d ago · 3 views

Memory Decoder at Scale: Scaling parametric long-term memory to 6.9B parameters

Researchers present Memory Decoder at Scale, a system that scales parametric long-term memory models up to 6.9B parameters and pretrains them on 300B tokens. To address the infeasibility of standard Faiss pipelines at this data scale, the authors implement a distributed indexing pipeline with sparse, batch-wise loading of kNN distributions.

arxiv arXiv cs.CL · 14d ago

Relay-OPD reduces training trajectory length by over 50% in on-policy distillation

Researchers introduce Relay On-Policy Distillation (Relay-OPD) to address prefix failure in standard on-policy distillation, where student errors lead to unreliable supervision. The method uses a teacher-student continuation asymmetry as a trigger for the teacher to briefly take over and redirect the trajectory before the student resumes.