A user reports deploying DeepSeek V4 Flash using the DSpark speculative decoding algorithm within SGLang on HGX-H200 hardware, comparing its performance against the Marlin MoE backend and EAGLE.

  • DSpark is 3.2x faster than EAGLE at batch size 1 and provides a 46% throughput increase at batch size 24.
  • While EAGLE maintains ~97-100% token acceptance, it drafts only 2 tokens per step; DSpark drafts 6 tokens per step with acceptance dropping to 88% at higher batch sizes.
  • Benchmarks show Marlin generally outperforms Flashinfer in Time to First Token (TTFT) for medium to high concurrency levels.
  • The deployment utilizes specific SGLang flags including `--speculative-algorithm DSPARK` and `--moe-runner-backend marlin`.