arxiv arXiv cs.CL · 53d ago · src: 54d ago · research

RegMix-D: Dynamic Data Mixing via Proxy Training Trajectories

from English

RegMix-D extends RegMix by leveraging full loss trajectories from proxy runs to dynamically select data mixtures. It outperforms RegMix and DoReMi across 13 downstream tasks, achieving superior results with just 128 proxy models—25% of RegMix's compute budget.

Importance 2/3 New harness with differentiators arXiv cs.CL OpenAI Google DeepMind Meta AI Evaluation & benchmarks Training data Training methods

Read original

arxiv arXiv cs.LG · 52d ago · 1 view

VIMPO: Critic-Free Policy Optimization for LLMs

VIMPO introduces a critic-free policy optimization method that derives a policy-implied value function from KL-regularized reinforcement learning. It enables verifiable reward incorporation without training a critic and outperforms GRPO on mathematical benchmarks, especially under noisy rewards.

arxiv arXiv cs.CL · 52d ago · 4 views

Bayesian Curriculum Learning on LLM Latent Manifolds

Manifold Bandits introduces Bayesian Manifold Curriculum (BMC), a framework that models problem sampling as a structured bandit problem in LLMs' latent space. BMC organizes tasks into a hierarchical tree and uses Bayesian learning to guide sampling, revealing tradeoffs between learning signal, task diversity, and evaluation relevance. Prioritizing difficulty alone fails to achieve strong downstream performance, underscoring the need for structure and type-aware sampling.

arxiv arXiv cs.LG · 52d ago

TAPO: Self-Distillation with Micro-Reflective Trajectories

TAPO advances self-distillation by constructing explicit micro-reflective trajectories that retain erroneous reasoning and insert natural-language diagnoses. These trajectories, derived from correct and incorrect model rollouts, provide fine-grained error corrections anchored in the model's own reasoning, improving both first-pass reasoning and error correction compared to GRPO.

arxiv arXiv cs.AI · 53d ago · 1 view

Self-Conditioned Credit Assignment for RL with Verifiable Rewards

SC-GRPO uses per-token KL divergence from self-conditioned trajectories to weight gradients in reinforcement learning. It outperforms GRPO by 8.1% and DAPO by 5.9% across math, code, and agentic tasks, with superior out-of-distribution performance and better results than OPD.

arxiv arXiv cs.CL · 53d ago

ConSA: Learnable Sparsity Control in Hybrid Attention

ConSA introduces a framework that learns optimal full vs. sliding-window attention allocation using L0 regularization and augmented Lagrangian constraints. It outperforms rule-based methods, with SWA placed in bottom layers and FA concentrated in middle-layer blocks, a pattern consistent across model scales and sparsity levels.