A study challenges the prevailing input-output approach to LLM-as-Judge scoring bias by demonstrating that biases are encoded in the model's hidden activation geometry. The authors analyze seven judges across nine benchmarks and identify a low-dimensional, type-specific subspace where biased inputs are displaced.
- Baseline judging inputs occupy a tight activation manifold, while biased inputs shift along a subspace that sharpens with network depth.
- Steering hidden states along this specific subspace causally drives scoring in both directions, whereas random directions produce negligible effects.
- A linear projection onto these bias-direction features accurately anticipates judge failures on three unseen benchmarks, outperforming text-based methods.
Reading bias as activation geometry unifies geometric structure, causal control, and operational prediction within a single framework for understanding LLM judging behavior.