github llama.cpp · 1h ago · inference

llama.cpp b10762 adds DeepSeek-V4-Flash-Vision-Exp support

from English

The llama.cpp project has released build b10762, which introduces support for the DeepSeek-V4-Flash-Vision-Exp model. This update also includes handling of minimum and maximum token counts from the command line interface.

Key changes in this release include:

Added mtmd support for DeepSeek-V4-Flash-Vision-Exp.
Implemented handling for min/max token counts via CLI.
Switched to using GGML_ROPE_TYPE_VISION.
Corrected token count logic and removed debugging code.

The release provides binaries for macOS, Linux, Windows, Android, and iOS across various hardware backends including CPU, CUDA, ROCm, Vulkan, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10766 adds DeepSeek 4 vision support

The llama.cpp project has released build b10766, which includes a fix to correctly support input vision for the DeepSeek 4 model.

github llama.cpp · 9h ago · 1 view

llama.cpp b10758 adds MUL_MAT and MUL_MAT_ID fusion for Hexagon

The llama.cpp project released build b10758, introducing MUL_MAT and MUL_MAT_ID fusion capabilities for the Hexagon DSP. This update also includes several fixes to improve stability and performance on Qualcomm hardware.

github llama.cpp · 9h ago · 3 views

llama.cpp b10759 avoids KleidiAI buffer type init on dispatch

The llama.cpp project released build b10759, which includes a fix to avoid KleidiAI buffer type initialization on dispatch.

github llama.cpp · 11h ago · 1 view

llama.cpp b10756 restricts Vulkan shader_bfloat16 extension request

The llama.cpp project released build b10756, which modifies Vulkan backend behavior to only request the VK_KHR_shader_bfloat16 extension when it is supported by the hardware.

github llama.cpp · 11h ago · 1 view

llama.cpp b10757 improves Vulkan IQ3_S mat-vec performance for larger batch sizes

The llama.cpp project released build b10757, which includes a specific optimization for the Vulkan backend. This update handles larger batch sizes greater than 4 efficiently for IQ3_S matrix-vector multiplication when NUM_COLS is greater than 4.