Researchers demonstrate that task-relevant information in instruction prompts can be compressed into a single activation vector and re-injected into large language models, replacing the original token sequence. This is achieved using a learned weighted sum of activations extracted at an intermediate layer and injected at an early layer.
- The compressed vector preserves task-relevant information with an accuracy drop of under 2% relative to full prompt processing.
- Mid-layer representations transfer meaningfully to early layers, suggesting cross-layer compatibility in how information is encoded.
- A single activation vector encodes a quantifiable and recoverable amount of semantic information.
- A weighted sum of activations serves as a robust representation compressor.
This approach reduces per-query computation for fixed instruction prompts without requiring reprocessing of the original token sequence.