Current oversight of LLM agents relies on scalar risk scores, but this fails to capture whether an intervention improves outcomes. The paper introduces "intervention advantage" as the key metric, showing that action-conditioned control outperforms scalar routing across benchmarks, with significant regret reduction in interactive regimes. Calibration alone does not resolve the underlying mismatch in control performance.
LLM-Agent Oversight Must Shift from Calibration to Action-Conditioned Control
from English