github llama.cpp · 1h ago · inference

llama.cpp b10144 fixes stream routes for model names with slashes

from English

The llama.cpp b10144 release addresses critical issues in the server and UI regarding resumable streams, specifically fixing handling of model names containing slashes. It also resolves bugs where requests were not properly cancelled or resumed during model loading.

Refactors resumable stream routes to use the `conv_id` query string, allowing model names with slashes to survive encoding in router mode.
Cancels pending requests when their stream is stopped during model load by registering the conversation before the autoload wait.
Enables resuming a stream after a page reload during model load by detaching session requests from client sockets and returning 503 for retries.
Restores visibility of model load progress in the UI after a page refresh by persisting the pending stream state.

These changes ensure that conversations remain robust against client disconnections and URL encoding issues, preventing orphaned requests and lost sessions during the model loading phase.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 6h ago

llama.cpp b10142 adds preliminary MiniMax-M3 vision support

The llama.cpp project released build b10142, which introduces preliminary vision support for the MiniMax-M3 model. This update implements a text-only port that reuses existing components from MiniMax-M2, including GQA with per-head QK-norm and partial rotary, DeepSeek-V3 style experts, and swigluoai activation.

github llama.cpp · 3d ago · 1 view

llama.cpp b10142 adds preliminary MiniMax-M3 vision support

llama.cpp b10107 fixes Windows crash and provides binaries for multiple platforms

llama.cpp b10105 refactors mlock/mmap/directio into a single load-mode argument

llama.cpp b10099 improves NVFP4 W4A4 quantization on CUDA

llama.cpp b10098 updates Hexagon activation ops