This work introduces a unified conceptual framework for discrete denoising diffusion models (DDMs) that views them through the construction of their underlying discrete state space. It demonstrates how existing formulations, including transition-matrix, masking/absorbing-state, and score/ratio-based approaches, emerge as different instantiations of a common design space.
- The framework highlights how DDMs are fundamentally shaped by tokenization schemes, vocabulary topology, and domain-specific structural alphabets.
- It exposes common design trade-offs across training objectives, inference algorithms, scaling behavior, systems optimization, and evaluation protocols.
- The authors suggest several promising directions for future research based on these exposed trade-offs.