This study applies Large Language Models to forecasting tasks and uses sparse autoencoders to analyze their internal states, distinguishing between time-specific knowledge and generalizable patterns. The research identifies specific features associated with both time-aware reasoning and look-ahead-biased reasoning.
- Researchers identified features linked to time-aware reasoning and look-ahead bias in LLMs.
- Amplifying time-awareness features substantially reduced look-ahead bias on forecasting prompts.
- General reasoning performance was preserved while reducing bias through feature amplification.
- Steering candidate look-ahead-bias features did not produce a significant effect.
These results suggest that interpretable temporal features can be used to causally shift LLMs toward more historically grounded reasoning.