arxiv arXiv cs.CL · 1h ago · research

Relay-OPD reduces training trajectory length by over 50% in on-policy distillation

from English

Researchers introduce Relay On-Policy Distillation (Relay-OPD) to address prefix failure in standard on-policy distillation, where student errors lead to unreliable supervision. The method uses a teacher-student continuation asymmetry as a trigger for the teacher to briefly take over and redirect the trajectory before the student resumes.

Utilizes a Qwen3-4B-Instruct-2507 teacher with Qwen3-0.6B/1.7B-Non-Thinking students on eight mathematical reasoning benchmarks.
Achieves the best or second-best results on every benchmark, outperforming standard OPD by +5.73% and FastOPD by +1.49% on average for the 1.7B model.
Consistent gains are observed at the 0.6B scale, while training trajectory length is reduced by over 50%.

This approach concentrates intervention on critical early positions via a limited relay budget, limiting departure from the student policy while significantly improving efficiency and accuracy.

Importance 2/3 arXiv cs.CL Alibaba (Qwen) Research paper Training methods

Read original

arxiv arXiv cs.CL · 13d ago

Post-training alignment of Qwen-based small language models for medication leaflet generation

A study compares post-training methods including supervised fine-tuning (SFT), direct preference optimization (DPO), odds ratio preference optimization (ORPO), and group relative policy optimization (GRPO) on Qwen-based small language models for biomedical data-to-text generation.

arxiv arXiv cs.CL · 15d ago

UMoE realigns MoE expert pools for improved domain-specific fine-tuning

Researchers propose UMoE, a pipeline that realigns the expert pool of Mixture-of-Experts models to a target domain before supervised fine-tuning. The method prunes low-saliency experts and regrows the pool via perturbation-based expansion while preserving original parameter counts and inference costs.

arxiv arXiv cs.CL · 19d ago

Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech

This paper describes a system for the MLC-SLM 2026 Challenge that combines a modular speaker diarization front end with a challenge-adapted Qwen3-ASR-1.7B recognizer.

arxiv arXiv cs.CL · 21d ago · 2 views

Frozen-backbone study finds domain adaptation efficacy depends on pre-existing target-domain knowledge

A case study evaluating frozen embedding backbones like Qwen3-Embedding, RoBERTa-base, and FinBERT reveals that the benefit of explicit domain adaptation varies significantly based on the model's prior knowledge. The research trained lightweight MLP adapters using Domain-Adversarial Neural Networks (DANN), Maximum Mean Discrepancy (MMD), and Supervised Contrastive Learning (SCL) to transfer sentiment analysis from consumer reviews to movie reviews (SST-2) and financial news.

arxiv arXiv cs.CL · 22d ago · 1 view

Study finds aligned language models transiently commit to wrong answers before late correction

A study of 17 models across three families reveals that aligned language models often transiently commit to incorrect answers in mid-layers (25-90% depth) before being corrected by late-layer mechanisms. This "wrong-dip" is verified causally through activation transplantation and varies significantly based on alignment recipes and model scale.