This paper presents a mathematical formulation of thinking and perception, formally deriving slow thinking as a form of active perception. The authors propose "active lifting," a theory based on sampling latent sequences to reduce uncertainty at the maximum rate.
- The framework encompasses the design, training, and inference of slow thinking large language models.
- It derives a large design space containing static theory subspaces where models can be upgraded via representation and sampler hierarchies.
- Technical by-products include a three-stage pathway for improving slow thinking models and a unified approach to constructing encoders for all data modalities.
- The theory also addresses the prior formation of human-like visual representations and offers a possible solution to policy collapse.
The work characterizes the agency of perception and explains the emergence of slow thinking formats through an inference process with an internal time axis.