github llama.cpp · 1h ago · inference

llama.cpp b10216 adds 1D pooling support to the Vulkan backend

from English

The llama.cpp project released version b10216, which introduces full GGML_OP_POOL_1D support for the Vulkan backend. This update includes a new compute shader mirroring the existing pool2d implementation and fixes a crash that occurred when tensor dimensions resulted in negative end boundaries.

Added vk_op_pool1d_push_constants struct and pipeline_pool1d_f32 field for 1D pooling operations.
Implemented pool1d.compute shader to handle 1D pooling logic on Vulkan.
Fixed shader logic by registering pool1d_f32 and correcting tensor dimension indices and average pool scale.
Resolved a boundary crash issue when k0 < p0 and expanded test coverage for the fix.

This release enables users to utilize 1D pooling layers within models accelerated by Vulkan, with binaries available for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, ROCm, and other backends.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago

llama.cpp b10215 adds Windows Intel GPU driver version check to fix crashes

The llama.cpp project released build b10215, which introduces a driver version check for Windows Intel GPUs to mitigate crashing issues. This change addresses instability caused by specific driver versions, specifically fixing crashes starting from driver 32.0.101.8860.

github llama.cpp · 3h ago

llama.cpp b10213 adds support for rotated kv cache quantization

The llama.cpp project has released version b10213, introducing support for rotated key-value cache quantization. This update is available across a wide range of platforms and hardware accelerators.

media r/LocalLLaMA · 4h ago

Initial testing of DeepSeek v4 Flash shows significant improvements in UI/UX design capabilities

Initial testing of the DeepSeek v4 Flash model indicates notable enhancements in its user interface and user experience design skills. Despite these improvements, the model is noted to be token hungry.

github llama.cpp · 5h ago

llama.cpp b10212 loads MTP tensors only when used

The llama.cpp project released version b10212, which includes a change to load Multi-Token Prediction (MTP) tensors only if they are actually used. This update ensures that MTP tensors are skipped during loading for remaining models that support the feature but do not require it.

github llama.cpp · 5h ago

llama.cpp b10210 corrects accepted tokens during draft token replay

The llama.cpp project has released version b10210, which includes a correction to the server logic for handling accepted tokens when draft token replay is required.