User seeks dual GPU optimization tips for running Qwen 3.6 35B

A Reddit user with a server equipped with two RTX 5060 16GB GPUs and 80GB of ECC DDR4 is asking for configuration advice to optimize performance when splitting model weights across the cards.

• The setup utilizes PCIe 3.0, which the user notes may hinder tensor parallelism efficiency.
• Current benchmarks show approximately 3200 tokens per second for prompt processing and 100 tokens per second for generation with the Qwen 3.6 35B model.
• The smaller Qwen 27B model achieves about 600 tokens per second for prompt processing and 23 tokens per second for generation in a tensor split configuration.
• The user is currently using GGUF formats with llama.cpp but also has access to vLLM NVFP4 models.

The post aims to gather community recommendations on settings to improve throughput and latency when running large language models across dual GPU hardware.