Researchers propose MemDefrag, a training-free framework that addresses performance degradation in large language models caused by positional encoding misalignment during latent memory updates. By identifying a tracing signal in middle transformer layers, the method reorders and filters stored knowledge fragments to improve retention.
- MemDefrag utilizes a middle-layer tracing signal to rank, reorder, and filter memories, resolving issues with irrelevant fragments.
- It applies an informativeness-guided proportional forgetting mechanism when memory capacity is exceeded.
- The approach substantially outperforms MemoryLLM and M+ on knowledge retention, achieving 43.0% versus 17.4%/17.6% after 50 updates.
- The framework generalizes well across various LLMs and latent-memory variants while improving long-context benchmarks.
This method significantly enhances long-term memory capabilities in LLMs by effectively managing stored knowledge fragments without requiring additional training.