The llama.cpp project has released version 0.5.0, focusing on backend performance, broader model coverage, and more robust server operation. This update introduces support for new architectures like HRM-Text (DFM Mimir 1B) and MiMo-V2.6, while significantly enhancing computational efficiency through CUDA and Metal optimizations.
Microsoft Research challenges the assumption that physical AI inference must run exclusively on onboard robot GPUs, demonstrating that offloading to edge or cloud infrastructure significantly enhances mobile manipulation workloads. Their systematic study of robotics workloads reveals that relying on onboard compute limits performance, battery life, and scalability.
The llama.cpp project has released version b11147, which introduces a new binary kernel for OpenCL. This update specifically adds support for the A8 Q6_K non-MoE dp4a configuration.
The llama.cpp project released version b11140, which includes a CUDA optimization enabling sparse-fa for dsv4 prefill. This update addresses performance issues in the flash attention mask scanning process by templating kernels to allow compile-time loop unrolling.
The llama.cpp project released build b11138, which updates the Jinja template parser to handle unary plus and minus operators before variables.
The llama.cpp project released build b11136, which updates the llama-server to accept the OpenAI standard "video_url" content type for video inputs. Previously, the server only accepted a non-standard "input_video" type and rejected data URIs, causing compatibility issues with OpenAI-conformant clients.
The llama.cpp project released version b11132, introducing support for the Gemma 4 DSpark draft backbone. This update enables GGUF conversion and runtime capabilities for full-attention and SWA Gemma 4 drafts, including tied output weights and boolean backbone metadata.
The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.
The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.
Researchers introduce CliffCompaction, an autocompaction technique designed to reduce costs for long-horizon coding agents by up to 50% under a bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping compacted information faithful through truncation rather than rephrasing.
Researchers introduce CliffCompaction, an autocompaction technique designed for agents handling millions of tokens, which cuts costs by up to 50% under bounded context. The method maintains or improves performance on Terminal-Bench and achieves state-of-the-art results on KernelBench by keeping information faithful through truncation rather than rephrasing.
The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.
Parallel has integrated OpenAI's GPT-6 Astra into its AI agent infrastructure, achieving a 50% reduction in both research time and code costs compared to prior models. The company reports that the new model allows agents to complete complex knowledge work significantly faster while maintaining high-quality output.
The llama.cpp project released build b11111, which introduces Vulkan-based Intel Xe flash attention optimization kernels for split k paths on Xe-LPG Plus, Xe2, and Xe3 architectures.
The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.
The llama.cpp project released version b11104, introducing a new feature that allows the llama-server component to bind to multiple network addresses simultaneously.
The llama.cpp project released build b11102, which introduces support for converting MiMo-V2.6 models.
The llama.cpp project has released version b11101, which includes a build system fix allowing repeated find_package calls for the llama library.
Hugging Face's Transformers library now supports loading GGUF quantized models, leveraging the underlying ggml kernels to bring performance close to llama.cpp. This integration allows developers to run quantized checkpoints directly within the standard PyTorch-based API on supported hardware.
The llama.cpp project released build b11100, which addresses a specific parsing issue with the Muse Glimmer model. The primary change involves fixing the first parser error encountered during tool-call execution for this model.