github llama.cpp · 1h ago · inference

llama.cpp b10638 adds Vulkan cross_entropy_loss and releases binaries

from English

The llama.cpp project has released version b10638, which implements Vulkan support for the `cross_entropy_loss` and `cross_entropy_loss_back` functions.

The release includes pre-built binaries for macOS (Apple Silicon and Intel), Linux (Ubuntu x64, arm64, s390x), Windows (x64 and arm64), Android (arm64), and openEuler.
GPU acceleration is available via Vulkan, CUDA (12.4, 13.3, 13.4 preview), ROCm 7.14, OpenCL Adreno, SYCL, and OpenVINO across supported platforms.
macOS Apple Silicon builds with KleidiAI are currently disabled in this release.
An updated UI package is also provided alongside the core binaries.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 1h ago · 1 view Live

llama.cpp b10639 fixes Vulkan warptile warp size assumption

The llama.cpp project released build b10639, which includes a fix for the Vulkan backend where warptiles previously assumed warp sizes were less than or equal to 64. This change clamps the warp size to work around issues with larger warps.

media Hugging Face Forums · 2h ago · 1 view

Celestium Neon fixes SDXL OOM on 4GB GPUs by cleaning VRAM

Celestium Neon is a new tool designed to clean VRAM and enable Stable Diffusion XL (SDXL) inference on graphics cards with only 4GB of memory, such as the GTX 1650.

github llama.cpp · 5h ago · 1 view

llama.cpp b10636 disables npm UI build by default and cleans up CI

The llama.cpp project released version b10636, which changes the default behavior for building the web interface. The npm-based UI build is now disabled by default (LLAMA_BUILD_UI=OFF), with the UI instead embedded from local sources or prebuilt downloads.

github llama.cpp · 6h ago · 1 view

llama.cpp b10635 unblocks mmq for MoE on sm_60

The llama.cpp project released version b10635, which includes specific updates to CUDA support. The primary change is the unblocking of mixed-precision matrix multiplication (mmq) for Mixture of Experts (MoE) architectures on compute capability 6.0 (sm_60) GPUs.

media Hugging Face Forums · 8h ago · 1 view

EdgeFirst publishes Rust pipeline validating YOLO models on 20 edge platforms

EdgeFirst has published the EdgeFirst Model Zoo on Hugging Face, featuring YOLOv5, YOLOv8, YOLO11, and YOLO26 models validated across 20 distinct edge platforms. The validation relies on a single Rust pipeline that handles zero-copy DMA pre-processing, model decoding, and post-processing directly on the target hardware.