A study using EvalAwareBench tests whether language models use a single shared direction or multiple cue-specific detectors to identify evaluation contexts. The research toggled eight trigger factors independently across 1,298 prompts while holding underlying tasks and entities fixed.

  • Analysis of Qwen3 (0.6B–14B) and Gemma 3 (270M/1B) models reveals that per-cue directions sit at only ~20% of the measurable cosine reliability ceiling, indicating distinct detectors rather than one shared axis.
  • Explicit evaluation disclosure is the primary driver, with a cosine similarity of ~0.89 to the global direction; removing it alone costs 36–57% of the eval/deploy signal.
  • Stimulus artificiality reads at chance (AUROC 0.52) on the global direction, even though the model represents it elsewhere with a dedicated direction of 0.68.
  • Evaluation framing does not improve capability but significantly changes willingness to answer, with mean top-letter log-prob shifting from -16.68 in deployment to -3.90 in evaluation at 14B.

The findings suggest that evaluation awareness is a bundle of cue-specific detectors rather than a single global axis, with explicit disclosure being the dominant factor.