The authors introduce Latent Memory Palace (LMP), a method that enables reasoning for continuous control policies by organizing information in an autoregressive latent space. The approach formulates this reasoning as variational inference with an autoregressive latent distribution and derives a latent-space reinforcement learning technique to optimize its variational lower bound.
- The resulting policy, LMP-$\pi$, achieves strong empirical performance in simulation and real-world domains while exhibiting interpretable, adaptive allocation of test-time compute.
- The framework also yields a variable-length action tokenizer, LMP-$\texttt{tok}$, which significantly improves the performance of downstream autoregressive policies.
These results present a new perspective on latent reasoning for control through the lens of variational inference.