Large language models use non-contiguous, redundant paths to retrieve factual attributes. These paths often skip layers and involve multiple equivalent routes, indicating distributed and redundant knowledge computation, challenging current understanding of LLM knowledge storage and retrieval.
Factual Retrieval in LLMs Is Non-Contiguous and Redundant
from English