Decoding hidden computation across filler tokens in frontier LLMs

Research reveals that frontier models like DeepSeek V3 and Kimi K2 perform multi-step reasoning over content-free filler tokens without visible chain-of-thought. This creates a behavioral oversight limit case where surface tokens carry no information about the underlying reasoning.

• Attention routes questions through filler regions to answers, with logit-lens readouts showing fact retrieval in early layers and composition in late layers.
• KV-cache transplants at filler positions causally swap outputs between examples.
• An unsupervised decoding pipeline recovers intermediate values with 80-95% accuracy across four task families without ground-truth labels or training.

The findings suggest that monitorability is a property of the model's full computational trace rather than just its surface tokens.