github llama.cpp · just now · inference

llama.cpp b10587 adds PAD_REFLECT_1D to Vulkan backend

from English

The llama.cpp project released build b10587, which introduces the GGML_OP_PAD_REFLECT_1D operation to the Vulkan backend. This update implements the necessary GLSL compute shader and registers it for SPIR-V compilation within the ggml-vulkan module.

Implemented PAD_REFLECT_1D logic in pad_reflect_1d.comp and updated vulkan-shaders-gen.cpp for registration.
Updated ggml-vulkan.cpp to handle constants, pipeline creation, dispatch functions, and debug validation for the new operation.
Validated correctness on Intel Iris Xe with Vulkan 1.4 and Mesa 25.2.8, passing all test cases.
Provided performance benchmarks showing execution times ranging from 5.38 us/run to 158.31 us/run across various tensor shapes.

This release enables the use of reflection padding in Vulkan-accelerated inference tasks and provides binaries for macOS, Linux, Windows, Android, and openEuler platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 5h ago · 1 view

llama.cpp b10584 fixes draft context sizing to prevent server 500 errors

The llama.cpp project released build b10584, addressing a critical issue where the draft context size did not match the target context, causing server requests to fail with 500 errors.

media Hugging Face Forums · 8h ago · 1 view

Xyntetik Runner trains LoRA directly through quantized GGUF weights

Joakimpalm-Zen has updated Xyntetik Runner to train LoRA adapters directly using the same quantized GGUF weights it serves for inference, eliminating the need for a separate FP16 training copy or runtime.

github llama.cpp · 10h ago · 1 view

llama.cpp b10581 adds DSpark support for bailinmoe3

The llama.cpp project has released build b10581, which introduces support for the DSpark backend to enable the bailinmoe3 model.

github llama.cpp · 12h ago · 1 view

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.

github llama.cpp · 13h ago · 1 view

llama.cpp b10576 adds Q2_K reordered MMVQ and ESIMD kernels for SYCL

The llama.cpp project has released build b10576, which re-adds Q2_K reordered MMVQ and ESIMD kernels for SYCL support. This change effectively reverts a previous commit that had removed these specific kernel implementations.