User seeks advice on building an offline LLM rig with dual Xeon CPUs and V100 GPU

A user in Russia is planning to build a fully offline AI rig due to anticipated internet restrictions and is seeking hardware and performance optimization advice. The proposed configuration includes two Intel Xeon 2683v4 CPUs, 256GB of DDR4 ECC RAM, and a single NVIDIA V100 16GB GPU, with an estimated budget of $1500.

• Target models include Deepseek V4 Flash (284B) and MiMo V2.5 (310B), with CPU-only inference estimated at 6-7 tokens per second for the former.
• The user reports that using ik_llama.cpp doubled prompt processing speed compared to standard llama.cpp in preliminary tests on a Qwen3.5 9B model.
• Optimization tips collected include configuring batch sizes, setting OMP_WAIT_POLICY=ACTIVE, and utilizing MTP models.
• Concerns remain regarding interprocessor communication latency and the availability of motherboards supporting eight memory channels for future GPU expansion.