A study reveals that biased LLM judges can silently disable the "skill retirement" mechanism in self-evolving agents, preventing them from discarding poor skills. This occurs because false-pass bias allows failures to slip through as passes, effectively turning off the curator that maintains skill library quality.
- Symmetric noise does not affect retirement, but false-pass bias disables contribution-based retirement past a sharp threshold.
- The mechanism failure is universal across domains and failure rates, sparing only near-zero-false-pass verifier-like graders.
- Downstream evaluation quality degrades only where corruption also starves skill synthesis; otherwise, performance holds steady.
- A defect-injection audit can determine if a judge operates on the safe side of the threshold before deployment.
The research highlights that disabled curators remain silent in aggregate metrics, posing a hidden risk to agent reliability.