The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.
The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.
The llama.cpp project has added support for DFlash speculative decoding with the HunyuanOCR model by exposing layer input tensors in the target graph. This change allows the draft model to read residual streams, enabling image requests to run successfully with a draft acceptance rate of approximately 0.5 and byte-identical OCR output compared to non-speculative runs.
The llama.cpp project released version b11104, introducing a new feature that allows the llama-server component to bind to multiple network addresses simultaneously.
The llama.cpp project released build b11102, which introduces support for converting MiMo-V2.6 models.
Hugging Face's Transformers library now supports loading GGUF quantized models, leveraging the underlying ggml kernels to bring performance close to llama.cpp. This integration allows developers to run quantized checkpoints directly within the standard PyTorch-based API on supported hardware.
Jun Kim, the creator and maintainer of oMLX, has joined Hugging Face to support the MLX community. This move aims to provide stability and faster development for the open-source project by transitioning it from a side job to a fully maintained and funded initiative.
A user is planning to build an uncensored, Greek-native large language model for local deployment using the ilsp/Llama-Krikri-8B-Instruct checkpoint. The proposed plan involves abliterating the model with Heretic (heretic-llm), converting it to GGUF Q4_K_M format, and running inference on a GMKtec EVO-X3 equipped with an AMD Ryzen AI MAX+ 395 processor via Vulkan.
The llama.cpp project released version b11090, which includes a fix for a sm_70 tile compilation error. The update generalizes the tile shape of the 5-argument load_ldmatrix function to resolve mismatches with Volta architectures.
The llama.cpp project has released build b11081, which includes several updates to the `test-llama-archs` testing infrastructure. Key changes involve making tensor data standard deviation configurable, expanding usage examples, and adding support for architecture regex patterns.
The llama.cpp project released build b11074, which includes a fix for JSON enum handling. The update adds common_json_value support for enum values and simplifies the code by delegating the enum constructor to the underlying type's constructor.
The llama.cpp project has released build b11067, which introduces a fused GDN + copy operation for the WebGPU backend.
The llama.cpp project released build b11060, which includes a fix for the Mamba model to ensure the time-step projection input is contiguous. This change addresses memory layout issues within the Mamba implementation.
The llama.cpp project released version b11054, which adds support for the TOP_K operator on Qualcomm Hexagon DSPs. This update includes optimizations for single-row processing and fixes for row partitioning.
The llama.cpp project has released version b11055, which introduces support for the GEGLU_QUICK activation function on Hexagon DSPs.
The llama.cpp project released version b11053, which includes a specific improvement to the server's startup log messages. This change enhances visibility into how models are loaded by replacing cryptic markers with explicit source tags.
The llama.cpp project has released build b11048, which introduces support for the new DSV4 HC op variants used by qwen4exp. This update includes specific implementations for hc_pre with per-element sigmoid gate and hc_post with identity mixing.
The NVIDIA version of Celestium CARE has been completed and made available as a professional tool for GTX and RTX graphics cards. It utilizes NVML to provide features such as VRAM cleaning, auto-cleaning schedules, and full telemetry monitoring.
The llama.cpp project released version b11046, which introduces OpenCL support for the `flash_attn_f32_f16_bin` kernel. This update also includes guarded prefill functionality for Flash Attention on OpenCL devices.
The llama.cpp project has released version b11042, which introduces a new OpenCL binary kernel for A8 Q6_K non-MoE models. This update also includes fixes for layout compatibility within the OpenCL backend.