The llama.cpp project released build b10181, which includes a fix in ggml-cuda to disable Matrix Multiplication Quantization (MMQ) on GPUs with less than 48 KiB of per-block shared memory. Previously, MMQ configurations assumed a minimum of 48 KiB, causing fatal errors on devices like NVIDIA Pascal GPUs and Moore Threads MTT S70 where the tile size exceeded available memory.

  • The fix adds a guard in `ggml_cuda_should_use_mmq()` to check shared memory budget before selecting MMQ tiles.
  • Devices with insufficient memory now fall back to the BLAS path during prefill, avoiding core dumps while maintaining token generation performance.
  • This change applies to any CUDA device reporting less than 48 KiB per-block shared memory, not just specific MUSA QY1 devices.

This update prevents crashes on older or specialized hardware with limited shared memory, ensuring stable inference by automatically using the compatible BLAS backend.