arxiv arXiv cs.CL · 2d ago · research

ROFT improves agentic models by fine-tuning on self-generated explanations

from English

Researchers investigate Retrospection-Only Fine-Tuning (ROFT), a minimal online procedure where an agent generates retrospective explanations of its own experiences and is fine-tuned using only the next-token prediction loss on those explanation tokens. This method requires no external teacher or reward-based policy update.

In software-engineering experiments with Qwen3.5-4B, ROFT reached 49.2% and 26.8% solve rates on SWE-bench Verified and Pro after 20 updates, outperforming GRPO's results after 40 updates. The approach also learns to solve tasks where all base-model attempts failed and encourages good actions by indirectly assigning credit.

These findings establish self-generated retrospections as useful training targets, demonstrating that learning to explain can improve learning to do.

Importance 2/3 arXiv cs.CL AI agents Training methods

Benchmarks

Benchmark	Model	Score
SWE-bench Verified	Qwen3.5-4B (with ROFT)	49.2%
SWE-bench Pro	Qwen3.5-4B (with ROFT)	26.8%

Read original

lab Microsoft Research Blog · just now Live

Microsoft Research system forecasts space weather risks for 66,935 U.S. substations

A machine learning pipeline developed at Microsoft Research generates location-specific geomagnetically induced current (GIC) risk estimates for 66,935 substations in the continental United States. The system combines solar-wind forecasts with local geological data to provide grid operators with 30 to 60 minutes of advance warning before specific risks appear.

arxiv arXiv cs.CL · 7d ago · 8 views

VHD-Play generates agentic RL environments from solved mechanisms

VHD-Play is a pipeline that generates diverse agentic reinforcement learning environments by sampling and solving mathematical models before rendering their decision processes as stateful tools. This approach ensures that executable dynamics and trajectory-scoring references are inherited directly from the solved model, addressing the misalignment issues common in existing generation pipelines.

arxiv arXiv cs.CL · 7d ago · 10 views

SkillGym internalizes human skills into LLMs for real-world problem solving

Researchers introduce SkillGym, a framework that transforms human-written agent skills into executable, verifiable training environments for large language model agents. The system utilizes a skill-to-task pipeline to instantiate concrete tasks and verify outcomes with code-based checkers.

arxiv arXiv cs.AI · 8d ago · 17 views

World State Generator aligns plans with synthetic worlds to improve agent success

The World State Generator (WSG) is a model that keeps language agent plans aligned with the rules of their execution environment by rewriting states after failures. It is trained on 226K trajectories extracted from seven synthetic domains where a program enforces rules and verifies goal reachability.

media Latent Space · 9d ago · 22 views

TypeSafe AI introduces Jev, a System One model trained with Reinforcement Learning for Calibrated Decisions

TypeSafe AI has launched Jev, a new class of "System One" models designed for production environments. The company's CEO and co-author of the InstructGPT paper, Diogo Almeida, argues that current frontier models have prioritized alignment and refusals over reliability, leading to issues like hallucinations and sycophancy.