github llama.cpp · 2h ago · inference

llama.cpp b10231 adds DSpark sidecar support

from English

The llama.cpp project released build b10231, introducing support for the DSpark speculative decoding sidecar. This update allows dspark files to resolve similarly to other speculative sidecars, with the -hfd tag applying to them and auto-selection prioritizing dspark over dflash due to its extra Markov head.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x with CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL backends), Android (arm64), Windows (CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler (x86 and aarch64 with ACL Graph).

This update expands the available hardware acceleration options for llama.cpp users across multiple operating systems and GPU architectures.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10235 adds SILU_BACK op for f32

The llama.cpp project has released version b10235, introducing a new SILU_BACK operation for f32 precision. This update includes the implementation of the silu_back operator and removes redundant assertions from the ggml-metal-ops.cpp file.

github llama.cpp · 1h ago · 1 view Live

llama.cpp b10232 implements DeepSeek V4 hyper-connections in Metal

The llama.cpp project released build b10232, which introduces support for DeepSeek V4 hyper-connections via the Metal backend. This update adds implementation of GGML_OP_DSV4_HC_COMB, GGML_OP_DSV4_HC_PRE, and GGML_OP_DSV4_HC_POST operations.

github llama.cpp · 6h ago

llama.cpp b10229 fixes OpenCL profiling data not being flushed

The llama.cpp project released version b10229, which includes a critical bugfix for the OpenCL backend. The update addresses an issue where profiling data failed to be written during program termination.

media Hugging Face Forums · 21h ago · 6 views

Hardik Kumar publishes HF inference pricing matrix showing up to 4.66x price spread

A new dataset published on Hugging Face provides a comprehensive pricing matrix for model inference providers, revealing significant cost disparities for identical models across different vendors.

github llama.cpp · 1d ago · 7 views

llama.cpp b10219 persists reasoning_content in chat history

The llama.cpp project released version b10219, which includes a fix for the CLI to persist reasoning content within the chat history. Previously, while `llama-cli` collected reasoning from the stream for display, it only stored assistant content in messages, preventing the `--reasoning-preserve` flag from re-injecting prior thoughts on later turns.