arxiv arXiv cs.CL · 1h ago · src: 3d ago · research

OpenForgeRL enables end-to-end training of harness-based agents in any environment

from English

OpenForgeRL is an open-source framework that allows researchers to train harness-based AI agents end-to-end using standard reinforcement learning stacks. It decouples training from inference by employing a lightweight proxy to record model calls as data and a Kubernetes orchestrator to manage rollouts in remote containers.

The system supports diverse environments, including tool/claw-based agents and multimodal GUI browser agents.
OpenForgeClaw achieves 31.7 pass^3 on ClawEval and 33.7 on QwenClawBench using only hundreds to a few thousand tasks.
OpenForgeGUI reaches 37.7 on OSWorld-Verified, 63.0 on Online-Mind2Web, and 72.3 on WebVoyager.
Both variants outperform open baselines of similar size and match or surpass larger models in GUI settings.
Analysis reveals that harness choice significantly impacts learning difficulty and that RL improves reliability metrics like self-verification.

The framework enables the study and improvement of agents directly within their deployment environments, demonstrating that reinforcement learning enhances agentic reliability despite remaining weaknesses in error recovery.

Importance 2/3 New harness with differentiators arXiv cs.CL OpenAI AI agents Training methods

Benchmarks

Benchmark	Model	Score
OSWorld	OpenForgeGUI	37.7%

Related articles

arxiv arXiv cs.AI · 3d ago · 3 views

OpenForgeRL enables end-to-end training of harness-based agents in any environment

OpenForgeRL is an open-source framework that allows researchers to train harness-based AI agents end-to-end using standard reinforcement learning stacks. It achieves this by decoupling training from inference through a lightweight proxy that records model calls as data and a Kubernetes orchestrator that manages remote container rollouts.

arxiv arXiv cs.LG · 38d ago

Marginal Advantage Accumulation for Memory-Driven Agent Self-Evolution

This paper introduces Marginal Advantage Accumulation (MAA), a post-processing architecture that addresses cross-batch inconsistency in memory-driven agent self-evolution. MAA formalizes alignment and comparability as structural conditions, uses differential signals and exponential moving average to accumulate signed evidence per operation, and ensures traceability via semantic identity merging. It outperforms batch-level baselines in 14 out of 16 settings and reduces token consumption by about 75%.

arxiv arXiv cs.LG · 38d ago · 1 view

LLM-based Hierarchical Control in Multi-Agent Games

A hierarchical system using a pretrained LLM to select RL skill policies outperforms flat RL in a 2v2 King of the Hill environment. It matches hand-crafted behavior tree performance in win rate and is perceived as more human-like by 60% of users, highlighting effective coordination and adaptability without manual rule design.

arxiv arXiv cs.CL · 39d ago

GraphPO: Graph-based Policy Optimization for Reasoning Models

GraphPO introduces a directed acyclic graph framework to represent reasoning rollouts, merging semantically equivalent paths to reduce redundant exploration. It assigns efficiency and correctness advantages to edges, improving inference efficiency and process supervision while reducing advantage-estimation variance. Experiments show GraphPO outperforms chain- and tree-based methods on three LLMs across reasoning and agentic search tasks under identical token or response budgets.

arxiv arXiv cs.AI · 39d ago

RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents

RODS addresses sample depletion in multi-turn tool-use RL by using reward variance to detect capability boundaries. It synthesizes new data in real time, matching structural complexity of boundary samples, and maintains a dynamic replay buffer that co-evolves with the policy. RODS achieves performance comparable to a 17K-sample offline pipeline with 20x fewer trajectories.