github llama.cpp · 2h ago · inference

llama.cpp b10273 resolves sampler context length and removes full-context windows

from English

The llama.cpp project released build b10273, addressing a critical issue where history-based samplers could not infer the correct context length during initialization due to backend-sampling constraints.

Resolves -1 to 1024 instead of ctx-len for samplers to fix initialization errors.
Sets a shared default of 64 for history-based samplers and removes the context_size parameter.
Provides binaries for macOS, Linux, Windows, Android, and openEuler across CPU, GPU (CUDA, Vulkan, ROCm, SYCL), and OpenVINO backends.

This update ensures stable sampler behavior during early construction phases when the complete llama_context is not yet available.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 4h ago

llama.cpp b10269 fixes dflash wo_a reshape on load

The llama.cpp project has released build b10269, which includes a fix for the dflash wo_a reshape issue during model loading.

media Hugging Face Forums · 7h ago

Open-source Decay-Gated O(N) Causal Linear Attention with fused Triton kernel

A developer has open-sourced a Decay-Gated O(N) Causal Linear Attention architecture featuring fused Triton/CUDA state accumulation kernels designed to bypass quadratic MHA bottlenecks.

github llama.cpp · 7h ago

llama.cpp b10262 adds GATED_LINEAR_ATTN to Vulkan backend

The llama.cpp project released build b10262, which introduces support for the GATED_LINEAR_ATTN operation in its Vulkan backend. This update also includes documentation updates for Vulkan operations and removes an unused GLA spec constant.

github llama.cpp · 10h ago

llama.cpp b10259 allows tensor reshape during load

The llama.cpp project has released build b10259, which introduces the ability to allow the reshape of tensors during the loading process.

github llama.cpp · 16h ago

llama.cpp b10256 parallelizes the SYCL non-contiguous concat kernel

The llama.cpp b10256 release includes a change to parallelize the non-contiguous concatenation kernel for SYCL devices. The update modifies the launch geometry from a single-lane work-group to one utilizing `SYCL_CONCAT_BLOCK_SIZE`.