DominoTree introduces conditional tree-structured drafting for speculative decoding

DominoTree is a training-free best-first draft tree method that leverages the released Domino drafter to accelerate large language model inference. It utilizes a GRU-based causal correction to make draft token distributions path-dependent, addressing limitations in factorized approaches like DDTree.

• Scores the draft tree using Domino's conditional, non-factorized correction along each root-to-node path.
• Restricts per-node correction to a candidate top-M set to maintain practical efficiency.
• Employs a GPU-native CUDA-graph builder that is bit-identical to the reference Python implementation.
• Achieves up to 6.6x speedup and a mean accepted length of 10.7 tokens on Qwen3-4B across eight benchmarks.
• Improves throughput by 9% to 10% on Qwen3-4B and up to 22% on Alpaca compared to the released Domino decoder.

DominoTree outperforms DDTree and CaDDTree at every tested temperature, including under greedy decoding, while maintaining aggregate gains over DFlash and Domino.