Researchers demonstrate that task-relevant information in instruction prompts can be compressed into a single activation vector by using a learned weighted sum of activations from an intermediate layer, which is then re-injected into an early layer of the target LLM.
- The method replaces the original token sequence with the compressed vector, incurring an accuracy drop of under 2% relative to full prompt processing.
- Analysis reveals that mid-layer representations transfer meaningfully to early layers, indicating cross-layer compatibility in information encoding.
- A single activation vector is shown to encode a quantifiable and recoverable amount of semantic information.
- The approach reduces per-query computation for fixed instruction prompts without requiring reprocessing of the original token sequence.