A user reports preliminary experiments running diffusion-based language models (dLLMs), specifically LLaDA2.1 and Sumi, on Apple Silicon hardware using a custom engine built for M1 and M2 Ultra devices.
- The baseline established throughput of 4.6–22.2 tok/s on M1 and 32.1–122.6 tok/s on M2 Ultra.
- Uniform 4-bit quantization was adopted as the default, resulting in a model footprint of ~9.57 GB with a low confident flip rate.
- Several optimizations were tested, including elastic cache (rejected), multi-block unmasking (provisionally accepted), and auto-speculation via S2D2 (accepted).
- TSCV voting improved GSM8K accuracy by 6–8 percentage points with zero CPU overhead, while Credit Decoding yielded negligible gains.
- Routing analysis revealed that distinct expert counts are lower than predicted, and quantization remains necessary as dequantization slows performance.
The author aims to build a functional diffusion engine for personal devices and gain deeper understanding of how these models operate.