The llama.cpp project released build b10751, which introduces a fused kernel for the MoE (Mixture of Experts) weighted expert reduction on CUDA. This optimization replaces the previous baseline that ran two physical kernels with a single weighted-reduction kernel to reduce intermediate global-memory traffic.
The llama.cpp project released build b10750, which refactors the key-value cell management to improve sequence position indexing and n-gram history lookups.
The llama.cpp project has released version b10742, which includes a new feature adding fast vector (fa-vec) tunings specifically for the Apple A18 Pro chip found in the MacBook Neo.
The llama.cpp project has released version b10739, which includes a specific performance optimization for Apple's M2 Max hardware. This update introduces fast attention vector (fa-vec) tuning tailored to the M2 Max's 30 GPU cores.
Hugging Face has released @huggingface/kernels, a library for loading and running optimized WebGPU kernels from the Hugging Face Hub, alongside an initial collection of 207 kernels. The release also introduces Fleet, an in-browser GPU benchmarking suite that crowdsources performance and correctness evidence from users' devices.
The llama.cpp project released version b10734, which includes an update to enable the Metal 4.0 tensor API for devices with M5+ or A19+ chips.
Turboderp has released significant updates to ExLlamaV3, introducing CPU offload capabilities for Mixture of Experts (MoE) experts. The update also includes support for the newly released GLM-5.3-Flash and Qwen3.8-Flash models.
The llama.cpp project released build b10724, which significantly improves the performance of restoring non-contiguous KV cache cells by batching scatter reads per contiguous run.
The llama.cpp project has released build b10721, which includes a critical fix for the WebGPU backend and expands hardware support on Windows.
The llama.cpp project released version b10718, which includes a significant update to its CUDA backend. The most notable change is the extension of Mixture of Experts (MOE) fusion to support speculative decoding (specdec), addressing previous limitations where MOE glu fusion and topk-router fusion were restricted to processing only one token at a time.
The llama.cpp project released build b10715, which includes a key optimization to the DFlash implementation. The DFlash encoder is now fused directly into the KV cache injection process during decoding.
The llama.cpp project released build b10714, which includes specific optimizations for the Vulkan backend targeting AMD Strix Halo hardware. The update adjusts matrix-vector multiplication row counts to improve performance during batched inference.
The llama.cpp project has released build b10706, introducing new functionality to the ggml backend. This update includes the addition of SWIGLU_CLAMP and a new Vulkan shader implementation.
The llama.cpp project released version b10707, which includes a performance optimization to the key-value (KV) cache handling. The update modifies the `for_each_token_in` function to stop scanning once all sequences within a specific cell have been seen, rather than iterating through all possible maximum sequences.
The llama.cpp project has released build b10688, which includes a pull request adding fast vector (fa-vec) tunings for the M2 architecture.
The llama.cpp project has updated its OpenCL backend to improve matrix multiplication performance on two generations of Adreno GPUs. The changes primarily target the X2E generation by defaulting the xmem F16xF32 GEMM path and optimizing handling for the A7X generation.
The llama.cpp project released version b10684, which includes a specific fix for the SYCL backend to make the `--fit` flag respect `--fit-target` more accurately.
The llama.cpp project released version b10670, which includes a specific optimization for Intel's Xe2 architecture. The update routes the quantized key-value (KV) decode operation to use the TILE backend exclusively for BMG hardware, while keeping the VEC backend on other architectures until they are validated.
The llama.cpp project released version b10669, which includes a performance optimization for the oneDNN SDPA path by binding the f16 KV cache in place.
The llama.cpp b10666 release expands the test-save-load-state suite to run across all architectures and resolves several critical bugs in state management and graph handling. Key changes include fixing dangling references in minimax-01, aligning chunk copying for on-device sequences, and correcting indexer head counts for deepseek4.