The llama.cpp project has increased the hoisted row-id limit for the Vulkan mul_mat_id operation from 256 to 1024 experts. This change addresses performance bottlenecks in models with large expert counts, such as Qwen3.8-Flash-Next, which previously ran on a slower code path due to shared array size constraints.
The update modifies the count_experts.comp shader to use a MAX_EXPERTS constant of 1024, growing shared arrays to 12 KiB while remaining within Vulkan's 16 KiB guarantee. On Strix Halo hardware with a batch size of 2048, expert matrix multiplications drop from 12.5 to 9.5 ms for iq3_s and from 14.0 to 7.5 ms for iq4_nl per operation. Prompt processing speed improves by approximately 19% at 8k tokens, and the MUL_MAT_ID backend tests pass with new 1024-expert cases.
This optimization allows llama.cpp to efficiently handle models exceeding the previous 512-expert threshold without performance degradation on Vulkan-compatible hardware.