The llama.cpp project released build b10709, which addresses a critical issue in the RPC subsystem regarding how tensor buffers are handled during cross-server communication.

  • The core change prevents the serialization of buffers belonging to other servers by only including remote buffer pointers when the buffer belongs to the RPC dispatcher receiving the graph.
  • A two-server regression test was added to verify correct behavior for cross-server tensor serialization.
  • Pre-built binaries are available for macOS (Apple Silicon and Intel), Linux (CPU, Vulkan, ROCm, OpenVINO, SYCL), Windows (CPU, CUDA 12/13, Vulkan, OpenVINO, SYCL, ROCm), Android, and iOS.

This fix ensures data integrity in distributed inference setups by preventing incorrect buffer sharing between distinct RPC servers.