github llama.cpp · 2d ago · inference

llama.cpp b11077 fixes router API key forwarding to child instances

from English

The llama.cpp b11077 release addresses a bug in router mode where the `--api-key-file` argument was incorrectly forwarded to router-spawned child instances. This caused clients using `--api-key` to receive 401 errors on chat completions and prevented internal router stream calls from working.

Unsets `LLAMA_ARG_API_KEY_FILE` in `unset_reserved_args()` so no API keys reach child instances.
Ensures all keys the router accepts work end-to-end without re-validation against file keys.
Prevents router internal stream calls, such as POST /v1/streams/lookup and DELETE /v1/stream, from being silently rejected with 401 errors.

This fix resolves issue #28820 by ensuring authentication remains the responsibility of the router rather than child processes.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1h ago · 11 views Live

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

arxiv arXiv cs.AI · 4h ago · 9 views

CliffCompaction reduces coding agent costs by 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed to reduce costs for long-horizon coding agents by up to 50% under a bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping compacted information faithful through truncation rather than rephrasing.

arxiv arXiv cs.LG · 5h ago · 9 views

CliffCompaction reduces long-horizon coding agent costs by up to 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed for agents handling millions of tokens, which cuts costs by up to 50% under bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping information faithful through truncation rather than rephrasing.

github llama.cpp · 6h ago · 13 views

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.