github llama.cpp · 1h ago · inference

llama.cpp b10331 fixes server get_info to report correct isolate working directory

from English

The llama.cpp project released build b10331, which includes a fix for the server's `get_info` endpoint. Previously, when no explicit current working directory was provided, the function incorrectly fell back to the server process's working directory, even if a tools runtime was configured.

The server now asks the isolate for its working directory when a tools runtime is active, ensuring the correct path is reported.
It retains the process working directory only when tools run directly on the host.
A redundant comment was removed from the codebase.

This change ensures that tool execution paths are accurately reflected in server info responses, preventing issues where named host paths did not match the isolate's actual environment.

Importance 1/3 Trust 1/3 llama.cpp Inference efficiency

Read original

github llama.cpp · 8h ago

llama.cpp b10330 fuses rms_norm, mul, and rope on CUDA

The llama.cpp project has released version b10330, introducing a CUDA optimization that fuses the rms_norm, multiplication, and RoPE operations into a single kernel. This change is accompanied by memory range checks for the fused operations and new test cases for broadcast weights.

github llama.cpp · 9h ago · 1 view

llama.cpp b10328 adds initial server tool isolation via Docker

The llama.cpp project has released build b10328, which introduces initial support for isolating tools within the server component using Docker. This update includes documentation and code adaptations to separate tool I/O sandboxes and rename relevant configuration flags.

github llama.cpp · 19h ago · 1 view

llama.cpp b10327 fixes CUDA thread/block count in quantized cpy kernel

The llama.cpp project released version b10327, which includes a fix for the thread and block count in quantized copy kernel launches on CUDA. This update addresses an issue identified in pull request #26731.

github llama.cpp · 1d ago

llama.cpp b10326 accounts for vocoder pass in timings

The llama.cpp project released build b10326, which modifies how timing measurements are calculated by including the vocoder pass.

github llama.cpp · 1d ago

llama.cpp b10321 fixes NORM/RMS_NORM for partial simdgroup row lengths

The llama.cpp project released version b10321, which includes a critical fix for GGML_OP_NORM and GGML_OP_RMS_NORM operations on Metal. The update resolves an issue where threadgroups sized to leave a partial SIMD group resulted in incorrect mean and variance calculations due to dropped partial sums.