The authors propose MILES (Modular Instruction Memory with LEarnable Selection), a framework designed to improve large language model reasoning at test time by dynamically accumulating reusable experience across sequentially arriving problems. Unlike existing methods that store rigid templates or use heuristic selection, MILES maintains modular memory units consisting of asymmetric pairs of sub-goal embeddings and sub-instructions, each associated with a learnable selection head.
- The framework employs a coarse-to-fine retrieval mechanism where the coarse level enables memory expansion and collects supervision from confident samples.
- The fine stage applies learned selection heads to rerank candidates and guide reasoning for uncertain samples.
- This approach allows for incremental memory expansion under realistic test-time constraints with limited supervision.
Extensive experiments demonstrate that MILES consistently matches or outperforms prior methods while achieving superior accuracy-efficiency tradeoffs, robustness, and transferability.