The llama.cpp project released build b10782, which addresses a critical issue in the CUDA backend for multi-GPU setups. The update allows concurrent streams per split and enables CUDA graph optimization to run correctly across multiple devices.
- Fixes a bug where CUDA events were incorrectly bound to the GPU that was current during creation, causing multi-GPU graph optimization to be skipped.
- Implements explicit device setting via `ggml_cuda_set_device` to ensure optimizations apply to the correct split per device.
- The fix is active only when `GGML_CUDA_GRAPH_OPT=1` is enabled; default behavior remains unchanged.
This change ensures that graph optimization runs once per split rather than being bypassed, allowing multi-GPU inference to utilize CUDA acceleration properly.