A pull request submitted to the ggml-org/llama.cpp repository introduces support for Tencent's Hy3 model, specifically the hy_v3 variant. This update enables the use of the model's multi-token-prediction head as a draft-mtp speculative target.

  • Adds support for Tencent's Hy3 (hy_v3 / HYV3ForCausalLM), a 299B MoE model with 80 layers plus one MTP layer.
  • Implements the multi-token-prediction head to function as a draft-mtp speculative decoding target within llama.cpp.