DeepSeek V4 Flash UD-Q2_K_XL quant runs on RTX 4090 with llama.cpp

A user reports running the DeepSeek V4 Flash model using unsloth's UD-Q2_K_XL quantization on an RTX 4090 with 128 GB DDR5 RAM via llama.cpp. The setup achieved a generation speed of 10.9 tokens per second, which the author considers reasonable for agentic tasks despite being slower than Qwen 3.6 27B.

• Pinning physical cores on an Intel Core Ultra 7 270k CPU doubled inference speed from 6.8 to 11 tok/s.
• Disabling flash attention (`-fa off`) and adjusting context quantization were necessary to prevent CUDA buffer overflow.
• The IQ4_NL quantization variant was too slow (8.1 t/s) and limited to a 10k context window.

The author suggests that with further optimization, this configuration could potentially replace Qwen 3.6 27B for users with 24GB GPUs.