TR-RAG uses teacher-regularized RL to fix cross-lingual RAG language drift

Researchers propose TR-RAG, a teacher-regularized reinforcement learning method that addresses language drift and unreliable evidence usage in English-evidence cross-lingual retrieval-augmented generation. The approach couples reward optimization with on-policy distillation, using a frozen teacher to provide prefix-wise reverse-KL anchors for answers sampled by a compact student.

• TR-RAG combines language consistency, character 3-gram recall, and LLM-judge scores into a reward decomposition for multilingual generation.
• The teacher anchor prevents large language-consistency collapses of up to ~27 percentage points seen in reward-only RL on in-domain languages.
• On distant out-of-distribution languages, the method improves evidence grounding where reward-only RL stalls at the base model's ceiling.
• The compact student sometimes surpasses its 70B teacher on character 3-gram recall across BioASQ-ENKB5, Hotpot-ENKB5, and MKQA benchmarks.

The authors consider this important because it stabilizes training and improves both language adherence and evidence-grounded correctness without requiring expensive full-model fine-tuning.