EcoSpec reduces active experts and speeds up decoding in sparse MoE models

Researchers propose EcoSpec, a cost-aware speculative decoding framework that incorporates predicted marginal expert activation cost into draft selection to address expert scattering in large-scale Mixture-of-Experts (MoE) models.

• EcoSpec uses a lightweight expert predictor and dynamic expert buffer to favor draft paths that preserve acceptance likelihood while reusing experts already covered by the current verification set.
• The method avoids modifying the target-model verification rule, instead optimizing the union of activated experts during parallel token verification.
• Evaluated on DeepSeek-V3.1 (671B), Qwen3-235B-A22B, and GPT-OSS-120B across reasoning, coding, question-answering, and dialogue benchmarks.
• Achieves up to 1.62x speedup by consistently reducing active expert footprints and improving end-to-end decoding speed.

These results demonstrate that accounting for expert activation cost is critical for efficient speculative decoding in large-scale MoE models.