github llama.cpp · 2h ago · inference

llama.cpp b10608 fixes video moov atom and SIGPIPE

from English

The llama.cpp project released build b10608, which includes a fix for the video module to correctly handle the moov atom at the end of files. This update also addresses SIGPIPE issues by adding handling for broken pipe cases on Windows.

mtmd: video: fix moov atom at the end of file
Fix SIGPIPE and handle broken pipe case on Windows

This release provides updated binaries for macOS, Linux, Windows, Android, and openEuler across various hardware backends including CPU, CUDA, ROCm, Vulkan, and OpenVINO.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · just now · 1 view Live

Mobile LLM assistant cuts token costs by 75% via caching and history compression

A mobile app developer reduced LLM API costs from four times the initial estimate to roughly a quarter of that amount by addressing two primary inefficiencies: naive full-history resending and redundant RAG context injection.

github llama.cpp · 1h ago · 6 views Live

llama.cpp b10610 shortens virtual device naming in CUDA and Metal

The llama.cpp project released version b10610, which includes a change to shorten virtual device naming within the ggml backend for both CUDA and Metal platforms.

github llama.cpp · 6h ago · 17 views

llama.cpp b10605 flattens mamba2 projections to dispatch GEMM

The llama.cpp project released build b10605, which includes a key optimization for the Mamba2 architecture. The update flattens the input and output projections of Mamba2 models to dispatch General Matrix Multiplication (GEMM) operations instead of General Matrix Vector (GEMV) operations.

media r/LocalLLaMA · 7h ago · 16 views

Qwen 3.8 27B scores 72.9 on Aider benchmark

The Qwen 3.8 27B model, running in FP8 with a 256K context window via vLLM, achieved a score of 72.9 on the Aider benchmark.

github llama.cpp · 7h ago · 11 views

llama.cpp b10604 adds Deepseek 4 DSV4 support and macOS/iOS binaries

The llama.cpp project has released version b10604, introducing support for the Deepseek 4 (DSV4) model format alongside new precompiled binaries for macOS and iOS.