DominoTree introduces conditional tree-structured drafting for speculative decoding

DominoTree is a training-free best-first draft tree method that combines the released Domino drafter with a GPU-native CUDA-graph builder to accelerate LLM inference. It scores draft trees using Domino's conditional, non-factorized GRU-based correction along each root-to-node path, restricted to a candidate top-M for efficiency.

• On Qwen3-4B across eight benchmarks, DominoTree achieves up to 6.6x speedup over autoregressive decoding and a mean accept length of up to 10.7 tokens per round.
• It wins throughput over the released Domino decoder by 9-10% overall on Qwen3-4B and up to +22% on Alpaca at every tested temperature.
• On Qwen3-8B, it maintains the highest accepted length and adds a decisive throughput win of +24% over DDTree at T=0.

The method provides significant speedup and acceptance rates compared to autoregressive decoding, DFlash, Domino, and DDTree/CaDDTree across various temperatures and models.