The llama.cpp project released build b9966, which addresses a performance issue in the -sm tensor feature. The update removes 29 regex recompilations per tensor per token that were previously occurring on the decode thread.
- The code previously rebuilt 29 regex patterns from scratch for every tensor of every token.
- The fix caches these patterns so they are built once and reused.
- This change reduces wasted CPU cycles on the decode thread without altering behavior.