Microsoft Research has introduced EvoLib, a framework that enables large language models to learn from their own experience during inference without requiring ground-truth labels or external feedback. Rather than storing raw experiences as static memories, EvoLib transforms them into reusable skills and reflective insights that are continually refined, consolidated, and reweighted.
- Consolidation merges new knowledge with similar existing entries to create more general and reusable units.
- A dynamic weighting mechanism updates the importance of each knowledge unit based on its immediate utility and long-term impact on future tasks.
- The system operates as a black-box solution applicable to any API-deployed language model without requiring model updates.
EvoLib consistently outperforms top retrieval-based memory approaches across mathematical reasoning, code generation, and long-horizon decision-making tasks. It converts test-time compute into performance gains more efficiently than existing methods and maintains stable performance regardless of the order in which heterogeneous tasks are encountered.