arxiv arXiv cs.AI · 1h ago · research

ISO introduces an RLVR-native optimization stack that fixes model weight spectra

from English

Researchers propose Isospectral Optimization (ISO), a framework for Reinforcement Learning with Verifiable Rewards (RLVR) that addresses the missing optimization layer by keeping model weight spectra fixed while optimizing input and output singular frames.

The approach includes ISO-Merger, which combines specialist capabilities into a single model without post-merge data or gradient updates, and ISO-Optimizer, which applies standard optimizers like AdamW to frame variables. On Qwen3-8B-Base, ISO-AdamW matched an aggregate accuracy of 0.495 in just 100 training steps compared to 270 for standard AdamW, eventually reaching 0.509 after 210 steps.

ISO provides a concrete method for post-training adaptation by inheriting the base model's spectrum and optimizing only the frames associated with reward-driven changes.

Importance 2/3 New harness with differentiators arXiv cs.AI Reasoning models Training methods

Read original

media r/LocalLLaMA · 10d ago

extGemma4-40_5B successfully extends Gemma 4 by adding capacity and healing

The author reports that the extGemma4-40_5B model, an extended version of Gemma 4, successfully overcomes previous failures in layer insertion by using a new initialization strategy. By initializing new layers as a blend of their neighbors rather than as no-ops, the model avoids signal starvation and allows for effective training.

arxiv arXiv cs.LG · 12d ago · 3 views

AdaPrefix-GRPO adapts trace prefixes to double GRPO accuracy on hard math

AdaPrefix-GRPO addresses the issue of vanishing gradients in Group Relative Policy Optimization (GRPO) by dynamically adjusting the length of correct solution prefixes prepended to training problems. By maintaining a success rate near 50%, the method ensures the gradient signal remains strong, eventually withdrawing assistance so the deployed model solves problems unaided.

arxiv arXiv cs.CL · 13d ago

AdaPrefix-GRPO adapts reference prefixes to double GRPO accuracy on hard math

The authors propose AdaPrefix-GRPO, a method that dynamically adjusts the length of a correct solution prefix during Group Relative Policy Optimization (GRPO) training. By treating prefix length as a feedback-controlled knob, the system maintains a success rate near 50% to maximize gradient signal before withdrawing assistance entirely.

arxiv arXiv cs.CL · 22d ago

LatentRevise: Learning from Zero-Hit Reasoning

The paper introduces LatentRevise, a first-order latent revision method designed to recover training signals in reinforcement learning with verifiable rewards (RLVR) for prompts where correct trajectories are rarely sampled. By optimizing the input embeddings of a reasoning prefix based on failed rollouts and gold answers, the method generates useful data from previously unproductive attempts.

arxiv arXiv cs.AI · 23d ago

ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

The paper introduces ScaleToT, a method that learns structured reasoning from a small subset of users and extends it to billions of low-activity users with sparse profiles. It combines a bounded entropy-guided Tree-of-Thought refinement with supervised fine-tuning and reward policy optimization to transfer reasoning capabilities without full LLM inference.