github llama.cpp · 1h ago · inference

llama.cpp b10642 adds token ID tracking to KV cell

from English

The llama.cpp project has released build b10642, which introduces a change to track token IDs within the key-value (KV) cell.

The core update involves adding token ID tracking to the KV cell implementation.
The `get_prev_tokens` function was removed from this specific commit and moved to the main pull request #27762.
Binaries are provided for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA 12/13, Vulkan, OpenCL, ROCm, SYCL), and openEuler.

This release provides updated binaries across multiple platforms with the internal KV cell modification.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 5h ago · 1 view

llama.cpp b10638 adds Vulkan cross_entropy_loss and releases binaries

The llama.cpp project has released version b10638, which implements Vulkan support for the `cross_entropy_loss` and `cross_entropy_loss_back` functions.

github llama.cpp · 6h ago · 2 views

llama.cpp b10639 fixes Vulkan warptile warp size assumption

The llama.cpp project released build b10639, which includes a fix for the Vulkan backend where warptiles previously assumed warp sizes were less than or equal to 64. This change clamps the warp size to work around issues with larger warps.

media Hugging Face Forums · 6h ago · 1 view

Celestium Neon fixes SDXL OOM on 4GB GPUs by cleaning VRAM

Celestium Neon is a new tool designed to clean VRAM and enable Stable Diffusion XL (SDXL) inference on graphics cards with only 4GB of memory, such as the GTX 1650.

github llama.cpp · 10h ago · 2 views

llama.cpp b10636 disables npm UI build by default and cleans up CI

The llama.cpp project released version b10636, which changes the default behavior for building the web interface. The npm-based UI build is now disabled by default (LLAMA_BUILD_UI=OFF), with the UI instead embedded from local sources or prebuilt downloads.

github llama.cpp · 11h ago · 1 view

llama.cpp b10635 unblocks mmq for MoE on sm_60

The llama.cpp project released version b10635, which includes specific updates to CUDA support. The primary change is the unblocking of mixed-precision matrix multiplication (mmq) for Mixture of Experts (MoE) architectures on compute capability 6.0 (sm_60) GPUs.