github llama.cpp · 1h ago · inference

llama.cpp b11306 fixes graph reallocation abort by toggling causal_attn

from English

The llama.cpp project released build b11306, which addresses a crash caused by graph shape changes during decoding. The update modifies the handling of the `causal_attn` flag to prevent reallocation failures under specific scheduling constraints.

After device decode, `causal_attn` is flipped off for half the ubatch and then back on for the remainder to ensure consistent node counts.
This change prevents the graph from reallocating at an unchanged size, which previously aborted execution when `GGML_SCHED_NO_REALLOC` was active.
The fix applies to decode architectures but is skipped for encode architectures.

The release includes binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, CUDA 12/13, ROCm 10.0, OpenVINO, SYCL, Snapdragon), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm 10.0), and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 2h ago · 1 view

llama.cpp b11301 fixes integer overflow in ggml/gguf

The llama.cpp project has released build b11301, which includes a fix for an integer overflow issue within the ggml and gguf components. This update addresses validation logic to prevent potential errors when handling zero-element tensors.

github llama.cpp · 2h ago · 1 view

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

The llama.cpp project released build b11302, which addresses a critical concurrency issue identified by ThreadSanitizer in the CI pipeline. The update resolves a data race within the sparse attention mechanism where multiple CPU threads were incorrectly writing to the same memory elements.

github llama.cpp · 3h ago · 2 views

llama.cpp b11301 fixes integer overflow in ggml/gguf

llama.cpp b11302 fixes ThreadSanitizer data race in sparse indexer

llama.cpp b11298 adds dflash support to convert and feature extraction

ThermaCompute AI releases CrashLens for local GPU crash log triage

llama.cpp b11284 fixes OpenVINO GET_ROWS on quantized weight views