The llama.cpp project released build b10724, which significantly improves the performance of restoring non-contiguous KV cache cells by batching scatter reads per contiguous run.
- Optimized state restoration for fragmented ring buffers by precomputing consecutive destination indices and using single copies instead of individual cell copies.
- Fixed an assertion failure in the on-device reader when save and restore tensor counts matched but chunking differed, falling back to a byte cursor copy.
- Added host and on-device tests verifying byte-identical state restoration for interleaved sequence cells.
This optimization reduces the number of memory copies required for large restores from over 1.3 million to just 224, cutting restore time from tens of seconds to under half a second.