github llama.cpp · just now · inference

llama.cpp b10593 fixes DeepseekV4 rollback and model loading

from English

The llama.cpp project has released build b10593, which addresses critical issues with the DeepseekV4 model architecture. The update specifically resolves a rollback failure that occurred during multi-sequence processing.

Fixes DeepseekV4 rollback handling for multi-sequence contexts.
Corrects general model loading procedures.
Makes pending rollback operations single-use to prevent state corruption.
Ensures cache is cleared only for the specific sequence ID during full loads.
Adds assertions for compression ratios and makes graph topology static.

This release improves stability for users running DeepseekV4 models with multiple sequences. It also provides updated binaries for macOS, Linux, Windows, Android, and iOS across various hardware backends including CPU, CUDA, ROCm, and Vulkan.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency Local inference

Read original

github llama.cpp · 20h ago · 2 views

llama.cpp b10584 fixes draft context sizing to prevent server 500 errors

The llama.cpp project released build b10584, addressing a critical issue where the draft context size did not match the target context, causing server requests to fail with 500 errors.

github llama.cpp · 2d ago · 3 views

llama.cpp b10568 uses ggml_rope_set_offset() for DeepSeek 2

The llama.cpp project released build b10568, which updates the model handling to use the `ggml_rope_set_offset()` function. This change is partially applied to support DeepSeek 2.

media Hugging Face Forums · 2d ago · 1 view

Celestium Engine adds VRAM optimizations and CPU fallback for SDXL

The Celestium Engine has introduced several upgrades to improve SDXL performance on low-VRAM hardware. These changes focus on memory management, stability, and fallback mechanisms to ensure consistent generation even on constrained systems.

github llama.cpp · 2d ago · 8 views

llama.cpp b10541 adds --mmproj-device argument for multimodal projects

The llama.cpp project released build b10541, introducing the `--mmproj-device` command-line argument to allow users to specify the device backend for loading multimodal projection files.

github llama.cpp · 2d ago · 2 views

llama.cpp b10545 fixes Metal matmul out-of-bounds reads for K not multiple of 32

The llama.cpp project released build b10545, which addresses a critical bug in the Metal backend's tensor API matrix multiplication kernel. The update prevents undefined behavior and corrupted results when processing tensors where the K dimension is not a multiple of 32.