The article introduces Weaver, a lightweight autoregressive adapter that constructs proposal trees from the top-K marginals of a factorized drafter to address the degradation in acceptance rates caused by independence assumptions.
- Weaver restores conditional dependencies between proposed tokens while avoiding a full-vocabulary projection.
- A rollback-free tree-verification algorithm is derived for models with Gated Delta Net layers, implemented via optimized CUDA kernels in SGLang.
- The approach achieves a 4.37-fold speedup over autoregressive decoding and outperforms a highly optimized DFlash baseline by 24.7%.
By combining these model and systems contributions, Weaver significantly increases the interactivity of autoregressive language models through speculative decoding.