Researchers propose a real-time monitoring system for large language models that uses an external model's verifier signal to trigger alarms when safety cannot be guaranteed. The system determines alarm decisions by thresholding the verifier output, with the threshold calibrated through risk control methods.
- The approach relies on simple thresholding of signals from an external model rather than complex sequential hypothesis testing.
- Experiments were conducted on mathematical reasoning and red teaming datasets to evaluate performance.
- The design is shown to be competitive with more advanced monitors based on sequential hypothesis testing.