github llama.cpp · 2h ago · inference

llama.cpp b11156 adds Ling 3.0 VL support via BailingMoeV3

from English

The llama.cpp project has released build b11156, which introduces support for the Ling 3.0 VL model. This update folds Ling 3.0 VL into the BailingMoeV3 architecture and ensures shared NORM rope lists remain intact when gating BailingMoe3 on mrope sections.

Adds Ling 3.0 VL model support by integrating it with the BailingMoeV3 architecture.
Maintains shared NORM rope list integrity during mrope section gating for BailingMoe3.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, CUDA, ROCm, OpenVINO, SYCL, Snapdragon), Android, Windows, and openEuler.

This release enables users to run Ling 3.0 VL models on a wide variety of hardware platforms and operating systems.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 12h ago · 3 views

llama.cpp v0.5.0 adds HRM-Text support, CUDA conv2d acceleration, and multi-address binding

The llama.cpp project has released version 0.5.0, focusing on backend performance, broader model coverage, and more robust server operation. This update introduces support for new architectures like HRM-Text (DFM Mimir 1B) and MiMo-V2.6, while significantly enhancing computational efficiency through CUDA and Metal optimizations.

github llama.cpp · 20h ago · 12 views

llama.cpp b11136 adds OpenAI video_url support to llama-server

The llama.cpp project released build b11136, which updates the llama-server to accept the OpenAI standard "video_url" content type for video inputs. Previously, the server only accepted a non-standard "input_video" type and rejected data URIs, causing compatibility issues with OpenAI-conformant clients.

github llama.cpp · 1d ago · 12 views

llama.cpp b11122 extends MMVQ GLU fusion and adds new SYCL fusions

The llama.cpp project has released version b11122, which includes significant updates to the SYCL backend. The release extends the MMVQ GLU fusion to support mixed quantization types and introduces new fusions for rms_norm+scale and ssm_conv+silu operations.

github llama.cpp · 1d ago · 15 views

llama.cpp b11120 hides Vulkan internal symbols to fix state destruction

The llama.cpp project released version b11120, which addresses a critical bug in the Vulkan backend by hiding internal symbols. This change prevents duplicate-dlopen state destruction issues that occurred when internal symbols were exported.

github llama.cpp · 1d ago · 13 views

llama.cpp b11118 introduces direct-mapped DMA cache for HVX FA mask handling

The llama.cpp project has released build b11118, which includes a new hex-dma feature introducing a direct-mapped DMA cache better suited for Hexagon Vector eXtended (HVX) FA mask handling.