media Hugging Face Forums · just now · open_models

Investigating whether failed agent traces are usable as fine-tuning data

from English

The author proposes a hypothesis that failed tool-using agent traces, such as wrong tool selections or malformed arguments, can serve as valuable fine-tuning data to train models away from these errors. The post seeks community feedback on whether training on corrected failure traces is effective or detrimental.

The author questions if "corrected" failure traces are lower quality because the fix is inferred rather than verified.
A key challenge identified is determining which failures have fixes derivable solely from the trace versus those requiring external ground truth.
The author built a CLI tool called trace2train to test this by detecting failures and outputting LLaMA-Factory SFT/DPO JSONL files.
The tool skips generating training pairs when the fix cannot be derived from the trace to avoid creating confidently-wrong labels.

The post aims to determine if using failed agent traces for fine-tuning is a viable strategy or a known-bad idea.

Importance 1/3 Trust 1/3 Hugging Face Forums AI agents Training methods

Read original

lab Microsoft Research Blog · 2d ago · 2 views

Microsoft Research introduces EvoLib for test-time learning with an evolving library

Microsoft Research has introduced EvoLib, a framework that enables large language models to learn from their own experience during inference without requiring ground-truth labels or external feedback. Rather than storing raw experiences as static memories, EvoLib transforms them into reusable skills and reflective insights that are continually refined, consolidated, and reweighted.

lab Microsoft Research Blog · 2d ago · 1 view

Microsoft releases Echoverse for training computer-use agents

Microsoft Research has released Echoverse, a collection of twelve deep synthetic environments designed to train computer-use agents. The initiative includes ten domain-specific worlds and two capability-focused worlds, all built to maintain coherent state and behavioral fidelity.

arxiv arXiv cs.CL · 6d ago

OpenForgeRL enables end-to-end training of harness-based agents in any environment

OpenForgeRL is an open-source framework that allows researchers to train harness-based AI agents end-to-end using standard reinforcement learning stacks. It decouples training from inference by employing a lightweight proxy to record model calls as data and a Kubernetes orchestrator to manage rollouts in remote containers.

arxiv arXiv cs.CL · 9d ago · 1 view

OpenForgeRL enables end-to-end training of harness-based agents in any environment

Researchers present OpenForgeRL, an open-source framework that allows for the end-to-end training of AI agents using complex inference harnesses like Claude Code and OpenClaw. The system decouples training from inference by employing a lightweight proxy to record model calls as data and a Kubernetes orchestrator to manage remote container rollouts.

arxiv arXiv cs.AI · 9d ago · 5 views

OpenForgeRL enables end-to-end training of harness-based agents in any environment

OpenForgeRL is an open-source framework that allows researchers to train harness-based AI agents end-to-end using standard reinforcement learning stacks. It achieves this by decoupling training from inference through a lightweight proxy that records model calls as data and a Kubernetes orchestrator that manages remote container rollouts.