arxiv arXiv cs.AI · 52d ago · research

Rubric-Conditioned Self-Distillation Framework

from English

Rubric-Conditioned Self-Distillation introduces a framework that uses structured rubrics to provide fine-grained, token-level feedback during self-distillation of reasoning language models. By conditioning teacher models on rubric-level criteria, it enables more precise credit assignment than scalar rewards, outperforming GRPO and OPSD by 1.0 and 0.9 points on average across science reasoning benchmarks.

Importance 3/3 Beats a top-lab benchmark New feature vs. leaders arXiv cs.AI Allen AI Microsoft Research OpenAI Evaluation & benchmarks Reasoning models Training methods

Read original

arxiv arXiv cs.CL · 52d ago

Rubric-Conditioned Self-Distillation Framework

arxiv arXiv cs.AI · 54d ago

Unified Causal-Origin Taxonomy for Distributional Shifts in RL

This paper introduces a unified causal-origin taxonomy that categorizes distributional shifts in reinforcement learning into internal, agent-driven, and external, environment-driven sources. It unifies ID/OOD generalization and non-stationary settings by framing shifts as structured changes in the agent-environment interaction process, using a POMDP decomposition and a shifted-time boundary perspective.

arxiv arXiv cs.LG · 54d ago · 1 view

HABC Improves RL Fine-Tuning of VLAs with Sparse Outcomes

Hierarchical Advantage-Weighted Behavior Cloning (HABC) enhances online RL fine-tuning of vision-language agents by using separate critic heads for viability and efficiency. It combines their outputs via a state-adaptive gate and applies per-transition weights, while intervention-aware credit assignment prevents supervision leakage. In real-robot experiments, HABC boosts success rates to 92%, 88%, and 38% on three bimanual tasks, surpassing SFT baselines of 36%, 44%, and 12%.

arxiv arXiv cs.CL · 51d ago · 1 view

H-RePlan: Hierarchical Recovery for Cross-Device Agent Systems

H-RePlan introduces a hierarchical replanning framework that separates device-local strategy recovery from global orchestrator replanning. It outperforms existing baselines by achieving higher completion and instruction adherence, with reduced token cost, through scope-aware recovery in multi-device agent systems.

arxiv arXiv cs.AI · 51d ago

Lean as Process-Verified Reward Oracle in RL for Theorem Proving

This work shows that Lean can serve as a symbolic process oracle, providing fine-grained, verified feedback during reinforcement learning. By parsing proof attempts into tactic sequences and using Lean's elaboration to mark sound steps and first failures, the system generates dense, type-theoretic reward signals. Experiments demonstrate tactic-level supervision outperforms outcome-only methods on benchmarks like MiniF2F and ProofNet, highlighting Lean's role as both evaluator and training reward source.