Researchers introduce a speculative execution method for LLM agents that equips the speculator with three online memory systems to learn from past trajectories, addressing the limitation of existing stateless speculators. The approach utilizes a contrastive transition table, episodic memory, and a confusion tracker to improve prediction quality over time.
- Memory-augmented speculation yields a 19--39% relative accuracy improvement on action prediction benchmarks.
- Observation prediction tasks see up to a 2.5x increase in performance with repetitive action spaces.
- Gains grow continuously as memory accumulates and generalize across speculator models of varying cost.
- All speculation runs during idle time, resulting in zero added wall-clock cost and identical actor trajectories.
This method provides lossless acceleration for LLM agents by leveraging historical data to enhance prediction accuracy without impacting execution speed or correctness.