github llama.cpp · 1h ago · inference

llama.cpp b10229 fixes OpenCL profiling data not being flushed

from English

The llama.cpp project released version b10229, which includes a critical bugfix for the OpenCL backend. The update addresses an issue where profiling data failed to be written during program termination.

The fix increments the `ref_count` in `ggml_backend_opencl_init()` to prevent it from becoming -1 in the `free()` method.
This ensures that profiling data is correctly flushed and saved when `GGML_OPENCL_PROFILING` is enabled.
Binaries are available for macOS, Linux, Windows, Android, and openEuler across CPU, CUDA, Vulkan, ROCm, OpenVINO, SYCL, and HIP backends.

This change ensures that users relying on OpenCL profiling can successfully capture and analyze performance metrics without data loss.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · 16h ago · 6 views

Hardik Kumar publishes HF inference pricing matrix showing up to 4.66x price spread

A new dataset published on Hugging Face provides a comprehensive pricing matrix for model inference providers, revealing significant cost disparities for identical models across different vendors.

github llama.cpp · 22h ago · 7 views

llama.cpp b10219 persists reasoning_content in chat history

The llama.cpp project released version b10219, which includes a fix for the CLI to persist reasoning content within the chat history. Previously, while `llama-cli` collected reasoning from the stream for display, it only stored assistant content in messages, preventing the `--reasoning-preserve` flag from re-injecting prior thoughts on later turns.

media r/LocalLLaMA · 1d ago · 3 views

Meituan releases LongCat-Flash-Lite-Sparse with 1M context window

Meituan has released the weights for LongCat-Flash-Lite-Sparse, a new variant of its LongCat-Flash-Lite model. The update introduces LongCat Sparse Attention (LSA) to replace dense MLA and extends native context length support to 1 million tokens.

github llama.cpp · 1d ago · 3 views

llama.cpp b10218 adds minicpmv46 downsample support

The llama.cpp project released build b10218, which includes the addition of minicpmv46 downsample support via pull request #25993. This update introduces a 4x ignore VIT merger and places the downsample mode inside the GGUF format.

github llama.cpp · 2d ago · 8 views

llama.cpp b10216 adds 1D pooling support to the Vulkan backend

The llama.cpp project released version b10216, which introduces full GGML_OP_POOL_1D support for the Vulkan backend. This update includes a new compute shader mirroring the existing pool2d implementation and fixes a crash that occurred when tensor dimensions resulted in negative end boundaries.