github llama.cpp · 1h ago · inference

llama.cpp b10105 refactors mlock/mmap/directio into a single load-mode argument

from English

The llama.cpp project released build b10105, which introduces a significant refactor of the command-line interface by consolidating separate memory management flags into a unified `load-mode` argument.

Overhauls mmap, mlock, and direct I/O settings into a single argument to simplify configuration.
Fixes bugs where variable behavior was incorrect and mlock was not activating as expected.
Updates llama-bench to sync argument parsing and correct column sizing issues.
Adds deprecation warnings when old and new flags are combined to guide users during the transition.
Includes general code cleanup, documentation updates via llama-gen-docs, and fixes for deprecated flag support.

This release helps users by streamlining memory configuration options and ensuring that critical features like mlock function correctly across supported platforms.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 12h ago

llama.cpp b10099 improves NVFP4 W4A4 quantization on CUDA

The llama.cpp project released version b10099, which includes a significant update to the CUDA backend for improved NVFP4 W4A4 activation quantization.

github llama.cpp · 14h ago

llama.cpp b10098 updates Hexagon activation ops

The llama.cpp project released version b10098, which includes an update to the Hexagon activation operations. This release provides binaries for macOS, Linux, Android, Windows, and openEuler across various CPU and GPU backends.

media Together AI Blog · 16h ago

Together launches Dedicated Model Inference with custom training beta

Together has released a significant update to its inference platform, introducing "Dedicated Model Inference" designed to give users complete control over performance, cost, and quality for open-weight models. The update also announces a closed beta for custom training capabilities, including full-weight and LoRA reinforcement learning.

github llama.cpp · 16h ago

llama.cpp b10094 infers speculative type from draft repo sidecars

The llama.cpp project released version b10094, introducing a feature that automatically detects the speculative decoding type from draft repository sidecars. Previously, when using the `-hfd` flag to point to a repository containing mtp-, dflash-, or eagle3- sidecars without explicitly providing `--spec-type`, the system would incorrectly resolve the draft as a full model.

github llama.cpp · 18h ago

llama.cpp b10093 fixes DeepSeek v4 template and adds drop_reasoning flag

The llama.cpp project has released build b10093, which includes specific updates to support DeepSeek v4 models. The release addresses template handling and introduces new configuration flags for this model family.