The llama.cpp project released build b10750, which refactors the key-value cell management to improve sequence position indexing and n-gram history lookups.

  • The `get_prev_tokens()` function was rebuilt to perform direct lookups instead of walking all used cells on every ubatch.
  • The internal index now stores `(pos, cell)` pairs in a `std::set`, allowing logarithmic time retrieval via the new `seq_pos_tok_le` method.
  • This change eliminates the need for the old window lookup and M-RoPE gap fallback logic.
  • Benchmarks on Qwen3.8-Flash-Next UD-Q4_K_XL at 71k context show a 4.9% increase in token generation speed (69.3 to 72.7 t/s) with unchanged prefill performance.

The optimization reduces computational overhead during inference, resulting in faster text generation for long-context scenarios.