TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding

Researchers propose Trajectory-Aware Commit Gating (TACG), a training-free decoder for diffusion language models that utilizes the trajectory of predictive distributions to improve token commitment decisions. TACG anchors token identities to the base posterior and uses trajectory-aware signals only to determine if a proposal is ready to commit, avoiding the conflation of confidence with readiness.

• The method combines Temporal Implicit Logits Guidance (TILG), which maintains an exponential moving average of past logits to contrast against current ones, with a History Gate (HG) that enforces short-term proposal persistence.
• A capped extra-promotion budget is used to create a stability-constrained commit rule without requiring auxiliary networks or extra forward passes.
• Evaluations on LLaDA, Dream, and LLaDA2-Mini across code (HumanEval, MBPP) and math (GSM8K, MATH500) benchmarks show that TACG typically improves or preserves accuracy while reducing denoising steps and increasing tokens per forward.

The approach allows diffusion language models to leverage their inherent trajectory data for more stable decoding without additional computational overhead during inference.