This survey addresses the memory-intensive nature of large language model (LLM) serving systems by focusing on system-aware key-value (KV) cache infrastructure, abbreviated as sKis. It organizes existing efforts into three dimensions: execution and scheduling (temporal), placement and migration (spatial), and representation and retention (structural).
The authors analyze cross-behavior co-design affinity and behavior-objective links to highlight future opportunities in KV cache design.
This work systematizes the rapidly evolving area, providing a foundation for understanding and innovating KV cache designs in modern LLM serving infrastructure.