A new project called vLLM Launcher has been released as a Windows desktop application designed to manage local large language model inference through WSL2. This tool allows users to control multiple engine backends, including vLLM, SGLang, and llama.cpp, from a single graphical interface.
The llama.cpp project released build b10331, which includes a fix for the server's `get_info` endpoint. Previously, when no explicit current working directory was provided, the function incorrectly fell back to the server process's working directory, even if a tools runtime was configured.
The llama.cpp project has released version b10330, introducing a CUDA optimization that fuses the rms_norm, multiplication, and RoPE operations into a single kernel. This change is accompanied by memory range checks for the fused operations and new test cases for broadcast weights.
The llama.cpp project has released build b10328, which introduces initial support for isolating tools within the server component using Docker. This update includes documentation and code adaptations to separate tool I/O sandboxes and rename relevant configuration flags.
The llama.cpp project released version b10327, which includes a fix for the thread and block count in quantized copy kernel launches on CUDA. This update addresses an issue identified in pull request #26731.
The llama.cpp project released build b10326, which modifies how timing measurements are calculated by including the vocoder pass.
The llama.cpp project released version b10321, which includes a critical fix for GGML_OP_NORM and GGML_OP_RMS_NORM operations on Metal. The update resolves an issue where threadgroups sized to leave a partial SIMD group resulted in incorrect mean and variance calculations due to dropped partial sums.
The llama.cpp project released build b10322, which includes a performance optimization that coalesces SSM_CONV window loads in the SYCL backend.
The llama.cpp project has released version b10313, which introduces a Least Recently Used (LRU) scheduler for its server component. This update includes handling for request coalescing and fixes for streaming cases.
The llama.cpp project released build b10311, which includes a fix for the Qwen3-TTS pipeline. The update resolves an issue where the model read the input utterance twice during generation.
The llama.cpp project has released version b10312, which includes a specific update to the server component. The key change addresses an issue where busy models were being evicted from the router.
The llama.cpp project released version b10307, which includes a fix for the SYCL GPU code to correctly parse UE4M3 scaling factors used in NVFP4 quantization.
The llama.cpp project has released build b10305, which introduces support for specific DSv4 operations within the SYCL backend. This update includes the implementation of LIGHTNING_INDEXER, DSV4_HC_COMB, DSV4_HC_POST, and DSV4_HC_PRE operators.
The llama.cpp b10306 release introduces performance optimizations for the SYCL backend, specifically targeting Gated Linear Unit (GLU) operations. The update adds a contiguous fast path for fused GLU ops and consolidates previously distinct kernels to share a common launcher.
The llama.cpp project has released build b10298, which introduces a new chunk save and load function for the mtmd module. This update also includes code cleanup and additional tests.
The llama.cpp project released build b10293, which includes onboarding AMD ROCm continuous integration for the gfx1151 architecture and resolves correctness issues on integrated RDNA3.5 GPUs.
The llama.cpp project has released version b10291, which includes critical fixes for the Vulkan backend alongside standard platform updates. The primary technical changes address submission batching size issues and introduce new diagnostic capabilities for driver errors.
The llama.cpp project released build b10290, introducing a new `ggml_build_forward_order` function to the ggml library. This change addresses a bug in the mtmd audio graph where using `ggml_build_forward_expand` as a pure ordering hint caused unselected branches to execute with stale inputs.
The llama.cpp b10289 release hardens the server's file_glob_search directory walk to prevent infinite loops on Windows junctions and improves error reporting for unreadable directories.
A new tool called QuantCheck highlights that the final pretraining checkpoint may not be the optimal choice for 4-bit quantization, as benchmarks can remain flat while fragility increases. The author observed this pattern on Pythia-160m, where the final checkpoint suffered approximately four times more quality damage than an earlier checkpoint of equal quality.