Dual GPU llama.cpp: Tensor parallel boosts decode, pipeline boosts prefill

A user benchmarks PCIe transfer and performance on a dual-GPU setup using llama.cpp with Qwen3.6-27B-UD-Q4_K_XL.gguf at 180k context. The test compares tensor parallel and pipeline parallel modes to evaluate bandwidth usage and throughput.

• Tensor parallel mode achieves a decode speed of 47 tokens per second versus 30 tokens per second for pipeline parallel.
• Pipeline parallel mode yields a higher prefill speed of 1250 tokens per second compared to 650 tokens per second in tensor parallel.
• PCIe bandwidth monitoring via nvidia-smi shows sustained transfer during decode in tensor parallel, while pipeline parallel exhibits variable prefill peaks and low sustained transfer.

The results suggest that using risers for connecting GPUs may not significantly bottleneck performance, supporting the viability of such hardware configurations for multi-GPU setups.