A study evaluates whether optimizer-driven gains compound over time by testing three agent-harness optimization methods on Terminal-Bench 2.0. The research compares GEPA, Meta Harness, and RELAI's Verifiable Continual Learning (RELAI-VCL) under identical budgets to determine if improvements persist as new tasks appear.
- GEPA's optimized agent performance transfers below the unoptimized baseline when new tasks are introduced.
- Meta Harness transfers well but fails to improve further after receiving a second optimization budget.
- RELAI-VCL is the only method that positively transfers to unseen tasks and continues improving, reaching the highest lifelong average pass rate of 76.4%.
- The authors observe that gains compound only when regression control is built into the optimization loop to prevent shortcut solutions.
The findings suggest that incorporating regression control provides an inductive bias against non-generalizing shortcuts, enabling sustained improvement in deployed agent settings.