A study reveals that large language model (LLM) judges tend to be overly generous when evaluating open-ended responses in no-reference settings, where no ground-truth answer is provided. The research demonstrates that these models often incorrectly credit wrong answers and that their judgments are highly sensitive to the presence of reference information.
- Calibration experiments assessed the judge model's knowledge of the task it was evaluating.
- Sensitivity experiments measured how performance changed based on the presence and positioning of reference answers.
- Across three languages, adding reference answer information flipped correct/incorrect decisions by as much as 85% in some settings.
- These reference-driven changes generally aligned with human annotations when compared to a subset of them.
The authors emphasize the need for calibrating LLM judges with reference-aware evaluation samples before using them reliably in reference-free setups.