github llama.cpp · 8h ago · inference

llama.cpp b11282 defines CUDA_ARCH for MUSA device passes

from English

The llama.cpp project released build b11282, which includes a fix to define the `CUDA_ARCH` macro for MUSA device passes. Previously, the MUSA vendor header failed to define this variable, causing architecture tests in shared ggml-cuda sources to evaluate to zero and resulting in empty kernel bodies.

The fix reports the newest architecture similar to the HIP backend while explicitly excluding NVIDIA-only features incompatible with MUSA.
`CUDA_ARCH` is now defined only for device passes to align with CUB's detection logic and nvcc behavior.
Redundant `defined(CUDA_ARCH)` checks in architecture comparisons were removed since the macro is undefined in host passes for CUDA and MUSA.

This change ensures that kernel bodies gated on architecture, such as the q8_0 -> f16 dequantization kernel, compile correctly for MUSA devices instead of expanding to empty bodies.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

media Hugging Face Forums · just now Live

ThermaCompute AI releases CrashLens for local GPU crash log triage

ThermaCompute AI has released CrashLens, a free, local tool designed to help engineers investigate GPU crashes by analyzing log files. The tool identifies known failure patterns within the logs and suggests specific next steps for investigation.

github llama.cpp · 6h ago · 2 views

llama.cpp b11284 fixes OpenVINO GET_ROWS on quantized weight views

The llama.cpp project released build b11284, which includes a fix for the OpenVINO backend to correctly handle GET_ROWS operations on quantized weight views.

github llama.cpp · 8h ago · 1 view

llama.cpp b11280 release reduces tensor allreduce sync with pinned host buffers

The llama.cpp project has released version b11280, which includes a change to reduce tensor allreduce synchronization by utilizing pinned host buffers.

github llama.cpp · 9h ago · 2 views

llama.cpp adds GLM-5.3-Flash support with k-pool and MTP fixes

The llama.cpp project has added initial support for the GLM-5.3-Flash (GLM5-Next) model architecture, including migration to hybrid-index memory and early Multi-Token Prediction (MTP) capabilities.

github llama.cpp · 11h ago · 2 views

llama.cpp b11273 adds classifier_pooling support for ModernBERT rerankers

The llama.cpp project released build b11273, which introduces support for the `classifier_pooling` method in reranking models, specifically targeting ModernBERT architectures.