Fine-tuning large language models often results in a "Knowing-Using Gap," where models memorize new facts but fail to apply them in downstream reasoning tasks. To investigate this, researchers introduced a technique called self-patching to monitor the spatial permeation dynamics of knowledge within the model.
The study identifies that memorized representations may exist internally but are not routed to computation-effective layers, supporting a knowledge-circuit misalignment hypothesis. Self-patching locates activation sites where relocating representations improves failed generalization cases. A heuristic strategy based on these findings recovers 58--75% of the potential improvement in generalization.
This diagnostic approach helps explain why memorized knowledge fails to generalize and offers a method to recover significant performance gains during model finetuning.