A demonstration shows that freezing key-value (KV) states from a guide text allows a Qwen2.5-3B-Instruct model to apply new instructions without including the guide in the live prompt. The method involves running a guide through the model once, freezing its KV cache, and then generating responses on top of that hidden prefix.
- In a math test with a made-up operator, the KV-guided model correctly computed results for unseen numbers, matching the performance of visible text injection.
- In an extraction task requiring itemized lists, the KV method produced correct outputs from the first token across multiple seeds, whereas visible injection required significant drift and no-guide attempts failed entirely.
- The approach is described as a hidden precompiled prefix rather than a computational mechanism, offering speed advantages over visible prompt padding.
This technique enables concept transfer and instruction following in large language models by leveraging frozen KV states instead of explicit textual context.