Researchers investigated whether internal representations in large language models provide a more direct window into calibration and faithfulness than chain-of-thought (CoT) reasoning. Working with Eternis-Forecaster 8B on OpenForesight, they trained representation-pooling probes on intermediate activations that achieved substantially better calibration compared to the model's standard outputs.
- Probes also functioned as lie detectors, tracking behavioral shifts and predicting change direction in 84% of cases even when CoT concealed perturbations.
- Evidence ablation showed that removing influential sources often changed forecasts while leaving reasoning traces untouched, highlighting a disconnect between output and process.
- Forced answering revealed forecasts are largely fixed before reasoning begins; routing questions by pre-set answer distribution spread saved 30-47% of tokens with no accuracy loss.
These results establish probing internal representations as a practical tool for calibrating, auditing, and triaging language model forecasters.