github llama.cpp · 1h ago · inference

llama.cpp b10635 unblocks mmq for MoE on sm_60

from English

The llama.cpp project released version b10635, which includes specific updates to CUDA support. The primary change is the unblocking of mixed-precision matrix multiplication (mmq) for Mixture of Experts (MoE) architectures on compute capability 6.0 (sm_60) GPUs.

CUDA: Unblocked mmq for MoE on sm_60.
CUDA: Duplicated mmq-config-pascal for dp4a and older architectures.
CUDA: Reduced occupancy on non-dp4a Pascal GPUs for Q2_K, Q4_K, Q5_K, and Q6_K quantization formats.

This release provides binaries for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), Android, and openEuler.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · just now · 1 view Live

llama.cpp b10636 disables npm UI build by default and cleans up CI

The llama.cpp project released version b10636, which changes the default behavior for building the web interface. The npm-based UI build is now disabled by default (LLAMA_BUILD_UI=OFF), with the UI instead embedded from local sources or prebuilt downloads.

github llama.cpp · 2d ago · 11 views

llama.cpp b10636 disables npm UI build by default and cleans up CI

llama.cpp b10604 adds Deepseek 4 DSV4 support and macOS/iOS binaries

llama.cpp b10598 uses pillow-accurate resize algo and corrects resize_algo for all models

llama.cpp b10593 fixes DeepseekV4 rollback and model loading

llama.cpp b10584 fixes draft context sizing to prevent server 500 errors