github llama.cpp · 1d ago · inference

llama.cpp b10872 fixes Jinja null left operand in template lookup

from English

The llama.cpp project released build b10872, which includes a fix for Jinja template handling. The update addresses an issue where templates defaulting an optional variable to none and testing its membership in a map previously hit an error.

Treats a null left operand of 'in' as a plain lookup, matching standard Jinja behavior.
Resolves errors in templates that test membership of undefined variables against maps.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm), and openEuler.

This change ensures template compatibility with standard Jinja semantics for undefined variables.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10889 release avoids V cache allocation for indexer

The llama.cpp project has released build b10889, which includes a memory optimization to avoid allocating the V cache for the indexer since it is not used.

github llama.cpp · 2h ago · 1 view

llama.cpp b10888 adds Vulkan command-buffer debug labels for GPU profilers

The llama.cpp project has released build b10888, which includes a new feature to add command-buffer debug labels for GPU profilers in the Vulkan backend.

github llama.cpp · 3h ago · 1 view

llama.cpp b10886 adds Q1_0 vector intrinsic support for s390x

The llama.cpp project released build b10886, which introduces Q1_0 vector intrinsic support for the s390x architecture. This update includes the implementation of `ggml_vec_dot_q1_0_q8_0` and updates documentation to reflect the new capability.

github llama.cpp · 17h ago · 1 view

llama.cpp b10883 fixes Vulkan matrix multiplication and restores Ampere optimizations

The llama.cpp project released build b10883, which includes significant updates to the Vulkan backend. The primary change involves using spec constants for matrix-matrix multiplication A-type operations to improve shader compilation and performance.

media r/LocalLLaMA · 20h ago · 1 view

gtrg55 releases INT4 quantized NVIDIA Cosmos3 64B for local image generation

A user has published code and weights for running NVIDIA's 64-billion parameter Cosmos3 model locally using INT4 quantization. The implementation supports both Text-to-Image and Image-to-Video tasks on Apple Silicon via MLX as well as CUDA.