github llama.cpp · 2h ago · inference

llama.cpp b10589 adds CUDA POOL_1D support

from English

The llama.cpp project has released version b10589, which introduces CUDA POOL_1D support via pull request #27573. This update also includes a fix for editorconfig compliance by adding a missing trailing newline.

The release provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (Ubuntu x64, arm64, s390x), Windows (x64 and arm64), and Android. Supported backends include CPU, CUDA 12/13, Vulkan, ROCm 7.14, OpenVINO, SYCL, and OpenCL Adreno.

This release enables users to run llama.cpp on a wide range of hardware architectures and operating systems with the new pooling capability.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 13h ago · 6 views

llama.cpp b10587 adds PAD_REFLECT_1D to Vulkan backend

The llama.cpp project released build b10587, which introduces the GGML_OP_PAD_REFLECT_1D operation to the Vulkan backend. This update implements the necessary GLSL compute shader and registers it for SPIR-V compilation within the ggml-vulkan module.

github llama.cpp · 18h ago · 2 views

llama.cpp b10584 fixes draft context sizing to prevent server 500 errors

The llama.cpp project released build b10584, addressing a critical issue where the draft context size did not match the target context, causing server requests to fail with 500 errors.

media Hugging Face Forums · 21h ago · 4 views

Xyntetik Runner trains LoRA directly through quantized GGUF weights

Joakimpalm-Zen has updated Xyntetik Runner to train LoRA adapters directly using the same quantized GGUF weights it serves for inference, eliminating the need for a separate FP16 training copy or runtime.

github llama.cpp · 23h ago · 2 views

llama.cpp b10581 adds DSpark support for bailinmoe3

The llama.cpp project has released build b10581, which introduces support for the DSpark backend to enable the bailinmoe3 model.

github llama.cpp · 1d ago · 1 view

ggml optimizes concat op by replacing per-element memcpy with row-level memcpy

The ggml library has optimized its concatenation operation by replacing per-element memory copies with row-level memcpy operations. This change is part of the b10578 release of llama.cpp, which includes various platform-specific binaries and updates to the underlying computation graph.