lab Tencent Hunyuan (HF) · 1d ago · releases

Tencent releases Simple Attention Sparsification checkpoints for Qwen3

from English

Tencent has released Simple Attention Sparsification (SAS) checkpoints that learn to rank and select KV blocks for each query in Qwen3 models. Unlike methods distilling dense attention scores, SAS adds continuous gates to selected blocks, enabling end-to-end optimization of context ranking with the language-modeling loss.

The release includes gate-router checkpoints for Qwen3-4B, Qwen3-8B, and Qwen3-14B models.
These are router-only weights that require a corresponding public Qwen3 base model and the seer_attn backend in a fork of SGLang.
The default sparse-attention setup uses a 2,048-token decode budget, but checkpoints can be evaluated with 1,024-, 2,048-, or 4,096-token budgets without retraining.

The checkpoints allow users to serve an OpenAI-compatible API using the seer_attn backend for efficient inference.

Importance 1/3 New harness with differentiators Trust 1/3 Tencent Hunyuan (HF) Tencent Inference efficiency Open weights

Read original

media MarkTechPost · 47d ago · 19 views

Tencent releases AngelSpec, a unified training framework for MTP and block-parallel speculative decoding

Tencent has open-sourced AngelSpec, a torch-native training framework that supports both autoregressive multi-token prediction (MTP) and block-parallel DFlash-family drafters for Hy3 models. The system treats workload heterogeneity as a primary design constraint, allowing it to specialize structure, training data, and verification depth based on specific use cases.

lab Tencent Hunyuan (HF) · 54d ago · 27 views

Tencent open-sources Hy-MT2 multilingual translation models and IFMTBench

Tencent has released the Hy-MT2 family of "fast-thinking" multilingual translation models, including 1.8B, 7B, and 30B-A3B (MoE) sizes, along with the IFMTBench benchmark for evaluating instruction-following capabilities.

media r/LocalLLaMA · 64d ago · 9 views

llama.cpp adds Hy3 support with MTP speculative decoding

A pull request submitted to the ggml-org/llama.cpp repository introduces support for Tencent's Hy3 model, specifically the hy_v3 variant. This update enables the use of the model's multi-token-prediction head as a draft-mtp speculative target.

arxiv arXiv cs.AI · 64d ago · 15 views

Deutsche Telekom releases Soofi S 30B-A3B for German and English

Deutsche Telekom has released Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts foundation model optimized for German and English. Built on the German Industrial AI Cloud, the model activates only 3 billion of its 30 billion parameters per token to maintain near-constant inference cache as context grows.

arxiv arXiv cs.LG · 64d ago · 19 views

Deutsche Telekom releases Soofi S 30B-A3B, an open-source German-English MoE model

Deutsche Telekom has released Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts hybrid Mamba Transformer foundation model designed for German and English. Built end-to-end on the German Industrial AI Cloud, the model activates only 3 billion of its 30 billion parameters per token to maintain near-constant inference cache as context grows.