github llama.cpp · 1d ago · inference

llama.cpp b11100 fixes Muse Glimmer tool-call parser error

from English

The llama.cpp project released build b11100, which addresses a specific parsing issue with the Muse Glimmer model. The primary change involves fixing the first parser error encountered during tool-call execution for this model.

Fixes Muse Glimmer tool-call first parser error
Adds test to verify the fix
Condenses patterns and removes test for trigger patterns

This release provides updated binaries for macOS, Linux, Windows, Android, and iOS across various hardware backends including CPU, CUDA, ROCm, Vulkan, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1h ago · 11 views Live

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

arxiv arXiv cs.AI · 4h ago · 9 views

CliffCompaction reduces coding agent costs by 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed to reduce costs for long-horizon coding agents by up to 50% under a bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping compacted information faithful through truncation rather than rephrasing.

arxiv arXiv cs.LG · 5h ago · 9 views

CliffCompaction reduces long-horizon coding agent costs by up to 50% while maintaining performance

Researchers introduce CliffCompaction, an autocompaction technique designed for agents handling millions of tokens, which cuts costs by up to 50% under bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping information faithful through truncation rather than rephrasing.

github llama.cpp · 6h ago · 13 views

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.