A study evaluates whether frontier models are necessary for verifying citations in deep-research systems, finding that cheaper judges remain competitive against gold labels.
- Researchers scored 8 off-the-shelf LLM judges from 3 model families on an adversarial long-form benchmark of 1,248 rubric decisions.
- GPT-5-mini achieved the strongest source-relevance pass-class F1 at 0.908, while factual support scores were statistically indistinguishable across models.
- Judges differed substantially in pass-rate drift and false positive/negative rates despite comparable scalar F1 scores.
The results indicate that calibrating the judge is a prerequisite for using citation rubrics as reward signals, but this calibration does not require the most expensive available model.