github llama.cpp · 1h ago · inference

llama.cpp b10203 adds SYCL support for q2_0 matrix multiplication

from English

The llama.cpp project has released version b10203, which introduces support for the q2_0 quantization format in SYCL-based matrix multiplication operations. This update expands the compatibility of the library for specific hardware acceleration scenarios.

Adds support for q2_0 in mul_mat via PR #26231.
Extends coverage to additional q2_0 cases within the SYCL backend.
Provides binaries for macOS (Apple Silicon and Intel), iOS, Linux (CPU, Vulkan, ROCm 7.2, OpenVINO, SYCL FP32/FP16), Android, Windows (CPU, OpenCL, CUDA 12/13, Vulkan, OpenVINO, SYCL, HIP), and openEuler.

The release enables users to utilize q2_0 quantization on SYCL-compatible devices, broadening the range of supported hardware configurations for efficient model inference.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · just now Live

llama.cpp b10206 enforces same K and V cache types for DeepSeek V4 and MLA models

The llama.cpp project has released version b10206, which introduces specific changes to how key-value caches are handled in attention mechanisms. The update ensures that the K and V cache types remain consistent for DeepSeek V4 models.

github llama.cpp · 2h ago

llama.cpp b10202 release with SYCL RMS_NORM fusion

The llama.cpp project has released version b10202, introducing a SYCL optimization that fuses the RMS_NORM and MUL operations.

github llama.cpp · 4h ago · 1 view

llama.cpp b10201 improves flash attention for quantized KV at long contexts

The llama.cpp project has released version b10201, which includes a key improvement to the ggml-webgpu backend. This update enhances flash attention performance when handling quantized key-value caches during long context generation.

media Latent Space · 7h ago

OpenAI cuts GPT-5.6 prices by up to 80% via self-optimization

OpenAI has announced significant price reductions for its GPT-5.6 models, driven by systemic efficiency improvements and recursive self-optimization techniques. The company reports that the cost of GPT-5.4 level intelligence has dropped approximately 13x in four months, with GPT-5.6 Luna prices slashed by 80% and Terra by 20%.

blog Simon Willison · 14h ago · 2 views

OpenAI cuts GPT-5.6 Luna price by 80% using Sol to optimize inference

OpenAI has significantly reduced the pricing for its GPT-5.6 model family, introducing a massive 80% price drop for GPT-5.6 Luna and a 20% reduction for GPT-5.6 Terra. These changes are enabled by GPT-5.6 Sol, which optimizes the model's forward pass and production kernels to improve efficiency.