github llama.cpp · 1h ago · inference

llama.cpp b10329 only shows working directory when a tool reads it

from English

The llama.cpp server and UI have been updated to conditionally display the working directory control. Previously, this chip appeared whenever any builtin tool was exposed, even if no tool actually used it.

Tools now declare whether they resolve paths against the working directory alongside their write permissions in the /tools listing.
The WebUI shows the working directory chip and enables the /cwd command only when at least one served tool is both enabled and capable of reading the directory.
This change prevents the UI from showing a useless control for servers started with tools like get_datetime or where all filesystem tools are disabled.

This update improves the user interface by ensuring that controls are only presented when they are functional, reducing clutter in configurations that do not require file system access.

Importance 1/3 Trust 1/3 llama.cpp Local inference

Read original

github llama.cpp · just now Live

llama.cpp b10330 fuses rms_norm, mul, and rope on CUDA

The llama.cpp project has released version b10330, introducing a CUDA optimization that fuses the rms_norm, multiplication, and RoPE operations into a single kernel. This change is accompanied by memory range checks for the fused operations and new test cases for broadcast weights.

github llama.cpp · 1h ago Live

llama.cpp b10328 adds initial server tool isolation via Docker

The llama.cpp project has released build b10328, which introduces initial support for isolating tools within the server component using Docker. This update includes documentation and code adaptations to separate tool I/O sandboxes and rename relevant configuration flags.

github llama.cpp · 11h ago · 1 view

llama.cpp b10330 fuses rms_norm, mul, and rope on CUDA

llama.cpp b10328 adds initial server tool isolation via Docker

llama.cpp b10327 fixes CUDA thread/block count in quantized cpy kernel

llama.cpp b10313 adds LRU scheduler to the server

llama.cpp b10312 release with server fix for busy models