Tencent has open-sourced AngelSpec, a torch-native training framework that supports both autoregressive multi-token prediction (MTP) and block-parallel DFlash-family drafters for Hy3 models. The system treats workload heterogeneity as a primary design constraint, allowing it to specialize structure, training data, and verification depth based on specific use cases.

  • AngelSpec addresses the train-inference mismatch in MTP by using a shared-parameter, multi-depth scheme where the physical block is autoregressively unrolled during training.
  • The framework employs target-model rollout and frozen backbone parameters to ensure the drafter approximates serving-time token choices without altering verification distributions.
  • The DFly block-diffusion architecture introduces hybrid target-conditioning and a predecessor-conditioned autoregressive head to improve acceptance rates in code and math domains.
  • On Qwen3-8B, DFly achieves an average mean accepted length of 5.41, outperforming DSpark (5.32), DFlash (4.57), and MTP (3.24).
  • On Hy3-A21B, DFly reaches a mean accepted length of 4.79, representing relative gains of 29.8% over DFlash and 59.7% over MTP.
  • The framework supports long-context training up to 128k tokens and includes pluggable interfaces for targets, objectives, and optimizers like Muon.

AngelSpec provides a unified pipeline for training six draft architectures, enabling significant speedups in live traffic throughput while maintaining high acceptance rates across diverse benchmarks.