A study demonstrates that frontier language models can perform consequential computation using semantically irrelevant filler tokens, creating a failure mode where reasoning is not visible in the output chain-of-thought. The research evaluated 13 models across three tasks and found that many benefit significantly from these tokens, with accuracy improvements of up to 13 percentage points.
- Filler tokens enable Claude Opus 4.5 to satisfy hidden modular arithmetic constraints without sacrificing primary task accuracy.
- Reinforcement learning gives Qwen3-235B strong preferences for specific filler token content, but neither RL nor supervised fine-tuning produces benefits that persist at test time.
- The performance benefit depends on which tokens are used and varies across different models.
These results indicate that frontier models already perform computation with no interpretable trace in their output tokens, demonstrating that invisible reasoning can serve objectives entirely invisible to chain-of-thought monitoring.