github llama.cpp · 1h ago · inference

llama.cpp b11377 honors json_schema in Ling 3.0 parser

from English

The llama.cpp project released build b11377, which updates the Ling 3.0 chat parser to properly handle `json_schema` constraints.

Previously, the Ling 3.0 grammar only supported tool calls and left `response_format` requests unconstrained. This release adds an eager response-format grammar path that takes precedence over tools.

The update requires a `<think>` block before JSON output when thinking is enabled and prevents trailing prose after the JSON response, fixing issue #29652.

This build includes binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, Snapdragon), Android, Windows (CPU, OpenCL, CUDA, Vulkan, OpenVINO, SYCL, ROCm), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b11375 fixes graph reallocation aborts under GGML_SCHED_NO_REALLOC

The llama.cpp project released build b11375, which addresses a critical issue where non-contiguous ubatch cells forced graph reallocations that would abort when the GGML_SCHED_NO_REALLOC flag was set.

github llama.cpp · 3h ago · 1 view

ggml-openvino updates to 2026.4.1 with MoE fusion and device listing fixes

The ggml-openvino backend for llama.cpp has been updated to version 2026.4.1, introducing performance optimizations, expanded operator support, and improved device enumeration.

github llama.cpp · 4h ago · 1 view

llama.cpp b11372 halves qwen4exp indexer memory and adds CUDA 4-head support

The llama.cpp b11372 release introduces optimizations for the qwen4exp model's lightning indexer, primarily halving the memory required for indexer scores during long context processing. This is achieved by computing head scores in place rather than materializing separate tensors for each head.

github llama.cpp · 10h ago · 2 views

llama.cpp b11370 fuses shared experts into MMVQ

The llama.cpp project has released build b11370, which includes a CUDA optimization to fuse shared experts into the Mixture of MoE with Vector Quantization (MMVQ) architecture. This change is part of pull request #29184 and also addresses buffer null checks and argument handling for stride_col_dst.

github llama.cpp · 13h ago · 2 views

llama.cpp b11368 adds probabilistic sampling for simple draft and MTP

The llama.cpp project released build b11368, which introduces probabilistic sampling for simple draft and MTP (Multi-Token Prediction) workflows. This update modifies the drafting mechanism to use probabilistic selection while verifying targets via rejection sampling.