github llama.cpp · 1h ago · inference

llama.cpp b10639 fixes Vulkan warptile warp size assumption

from English

The llama.cpp project released build b10639, which includes a fix for the Vulkan backend where warptiles previously assumed warp sizes were less than or equal to 64. This change clamps the warp size to work around issues with larger warps.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Android, Windows (CPU, CUDA, Vulkan, OpenCL, OpenVINO, SYCL, ROCm), and openEuler. It also includes the llama.cpp UI.

This update ensures compatibility with hardware configurations that utilize warp sizes larger than 64.

Importance 1/3 Trust 1/3 llama.cpp Hugging Face Inference efficiency

Read original

media Hugging Face Forums · 7h ago · 1 view

EdgeFirst publishes Rust pipeline validating YOLO models on 20 edge platforms

EdgeFirst has published the EdgeFirst Model Zoo on Hugging Face, featuring YOLOv5, YOLOv8, YOLO11, and YOLO26 models validated across 20 distinct edge platforms. The validation relies on a single Rust pipeline that handles zero-copy DMA pre-processing, model decoding, and post-processing directly on the target hardware.

lab Hugging Face Blog · 1d ago · 1 view

Papers with Code uses Hugging Face services for hybrid paper search

Papers with Code has implemented a hybrid search system combining keyword and semantic retrieval to help users find relevant AI research papers. The architecture leverages Hugging Face Jobs, Storage Buckets, and Inference Endpoints to manage the embedding pipeline and serve low-latency queries.

github llama.cpp · 1d ago · 21 views

llama.cpp 0.3.0 adds dots3-note model and tensor-split for DeepSeek 4

llama.cpp 0.3.0 introduces support for the dots3-note multimodal model with a new DSA-ISWA KV cache type, along with multi-token prediction (MTP) support for GLM-4.5-Air. The release also adds tensor-split capabilities for DeepSeek 4 and fixes multi-sequence rollback issues.

github llama.cpp · 2d ago · 4 views

llama.cpp b10615 adds per-device tuned flash-attn vec for Metal

The llama.cpp project released version b10615, introducing per-device tuned quantized (Q, NE) flash-attention vector operations for the Metal backend.

github llama.cpp · 2d ago · 1 view

llama.cpp b10614 adds Metal per-op source split and parallel compile

The llama.cpp b10614 release introduces a significant architectural change to the Metal backend by splitting operation sources and enabling parallel compilation. This update also includes several new kernel implementations and support for additional quantization formats.