A new proposal outlines an LLM memory structure that combines late chunking with attention-weighted edges to enable step-by-step traversal. The system leverages Jina AI's embedding model, which calculates attention between sentences, to derive edge weights from the document's internal processing.
- Late chunking preserves document-level context by tokenizing and embedding the whole text before deriving chunk vectors.
- Attention matrices from a forward pass define graph edges, with weights based on how much one chunk attended to another.
- Retrieval uses swarming traversal with a decaying budget, stopping when costs exceed limits rather than at fixed hop counts.
- Embedding updates are nudged by remaining budget, ensuring central nodes change more than distant ones.
This approach allows for dynamic context management where the relevance of retrieved information determines its influence on the memory state.