A new study identifies the "Knowing-Using Gap," a phenomenon where large language models memorize fine-tuned facts but fail to apply them in downstream reasoning tasks. The authors characterize this gap through accuracy deficits and temporal lags between memorization and generalization.
The researchers utilized a novel intervention technique called self-patching to monitor internal knowledge permeation dynamics during finetuning with unseen data. This method identifies specific activation locations where relocating representations significantly improves failed generalization cases, supporting the hypothesis that memorized representations exist but are not routed to computation-effective layers. To address this, they designed a heuristic strategy that recovers 58--75% of the potential generalization improvement.
The findings demonstrate that misalignment between stored knowledge and computational pathways is a key factor in finetuning failures, offering a diagnostic approach to enhance model utility across domains.