github llama.cpp · 2d ago · inference

llama.cpp b11080 allows regex in -o filter and stops FA vec slice logging

from English

The llama.cpp b11080 release updates the test backend operations to support regular expressions in the `-o` filter, allowing pattern matching for op names while preserving exact matches for plain names. It also modifies the `cont` tool to stop printing Fast Attention vector slice logs when they are not needed.

The `-o` filter now treats non-plain entries as regexes (e.g., "MUL_MAT.*") against op names, while plain names like "ADD" remain exact matches.
The help text for `tests/test-backend-ops` has been reformatted with separate sections for modes, options, and examples.
macOS, Linux, Windows, Android, and iOS binaries are provided for CPU, CUDA, ROCm, Vulkan, OpenVINO, and SYCL backends.

This release provides updated binaries across multiple platforms and improves test filtering flexibility.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1h ago · 11 views Live

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

arxiv arXiv cs.AI · 4h ago · 9 views

CliffCompaction reduces coding agent costs by 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed to reduce costs for long-horizon coding agents by up to 50% under a bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping compacted information faithful through truncation rather than rephrasing.

arxiv arXiv cs.LG · 5h ago · 9 views

CliffCompaction reduces long-horizon coding agent costs by up to 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed for agents handling millions of tokens, which cuts costs by up to 50% under bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping information faithful through truncation rather than rephrasing.

github llama.cpp · 6h ago · 13 views

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.