github llama.cpp · 2h ago · inference

llama.cpp b10305 adds SYCL support for DSv4 OPs

from English

The llama.cpp project has released build b10305, which introduces support for specific DSv4 operations within the SYCL backend. This update includes the implementation of LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE operators.

Support added for DSv4 OPs: LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE in the SYCL backend.
Updates to the ops.md documentation file.
Fixes applied to formatting issues within the codebase.

This release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu with CPU, Vulkan, ROCm 7.2, OpenVINO, and SYCL FP32/FP16), Android, Windows (CPU, OpenCL Adreno, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 1h ago Live

llama.cpp b10307 fixes SYCL NVFP4 parsing

The llama.cpp project released version b10307, which includes a fix for the SYCL GPU code to correctly parse UE4M3 scaling factors used in NVFP4 quantization.

github llama.cpp · 2h ago

llama.cpp b10306 adds SYCL GLU flat path and consolidates kernels

The llama.cpp b10306 release introduces performance optimizations for the SYCL backend, specifically targeting Gated Linear Unit (GLU) operations. The update adds a contiguous fast path for fused GLU ops and consolidates previously distinct kernels to share a common launcher.

github llama.cpp · 16h ago

llama.cpp b10298 adds mtmd chunk save/load function

The llama.cpp project has released build b10298, which introduces a new chunk save and load function for the mtmd module. This update also includes code cleanup and additional tests.

github llama.cpp · 1d ago · 3 views

llama.cpp b10293 adds ROCm CI for gfx1151 and fixes RDNA3.5 inference

The llama.cpp project released build b10293, which includes onboarding AMD ROCm continuous integration for the gfx1151 architecture and resolves correctness issues on integrated RDNA3.5 GPUs.

github llama.cpp · 1d ago · 1 view

llama.cpp b10291 fixes Vulkan submission batching and adds DeviceLost debug tools

The llama.cpp project has released version b10291, which includes critical fixes for the Vulkan backend alongside standard platform updates. The primary technical changes address submission batching size issues and introduce new diagnostic capabilities for driver errors.