arxiv arXiv cs.CL · 1h ago · research

PIVOT shares prefix scans across query groups to accelerate token-level sparse attention

from English

PIVOT (Proxy Indexing Via One full-prefix Traversal) is a training-free, drop-in replacement for the DeepSeek Sparse Attention (DSA) indexer that reduces computational redundancy by sharing one prefix scan across a group of nearby queries. Instead of scoring every preceding token for each query individually, PIVOT aggregates the group into a single proxy query to obtain a candidate set, from which top-k tokens are selected for each query.

Two variants are offered: PIVOT-Reuse shares the proxy top-k for maximum speed, while PIVOT-Refine re-scores candidates with individual indexers to match dense indexer fidelity.
The algorithm handles both prefill and decode phases by forming groups of consecutive queries or multi-token prediction steps respectively.

On DeepSeek-V3.2 and GLM-5.1 across LongBench and RULER, PIVOT matches the accuracy of the dense DSA indexer while accelerating it by up to 4x and reducing end-to-end latency by up to 1.6x at long context.

Importance 2/3 arXiv cs.CL DeepSeek Zhipu AI Inference efficiency Research paper

Read original

arxiv arXiv cs.CL · 21d ago

DSpark introduces confidence-scheduled speculative decoding to accelerate LLM inference

DSpark is a speculative decoding framework designed to accelerate Large Language Model (LLM) inference by combining semi-autoregressive draft generation with adaptive, load-aware verification. It addresses the acceptance decay and throughput degradation issues found in existing parallel drafters.

media r/LocalLLaMA · 25d ago · 3 views

GLM-5.2 NVFP4 on four DGX Sparks — the MTP mystery is solved, and it's now ~24 tok/s at 128K context

A follow-up investigation into running GLM-5.2 NVFP4 on four DGX Spark nodes resolves a previous performance bottleneck where high acceptance rates were impossible at 128K context.

media r/LocalLLaMA · 25d ago · 1 view

Deepseek drops another HUGE breakthrough - DSpark. Waaay faster than MTP [Video explaining it]

A Reddit user shared a video link regarding a new DeepSeek breakthrough called DSpark, which is described as significantly faster than MTP.

arxiv arXiv cs.CL · 31d ago · 2 views

HyperDFlash: MHC-Aligned Block Speculative Decoding with Gated Residual Reduction

HyperDFlash is a block-parallel speculative decoding framework designed to address feature misalignment issues when adapting DFlash to DeepSeek-V4's multi-hyper-connection (MHC) architecture. The authors propose two key optimizations: using pre-collapse residual states for conditioning and replacing the generic linear compressor with a lightweight gated residual reducer inherited from the model's hyper-connection head.

arxiv arXiv cs.CL · 3h ago

Zing framework improves LLM social intelligence via SoMBench benchmark and Actio grounding

The report introduces Zing, an integrated framework designed to enhance the social intelligence of large language models by measuring, internalizing, and grounding social capabilities. The authors present SoMBench, a psychology-grounded benchmark spanning 17 secondary dimensions and 3,481 expert-verified instances, which reveals that current LLMs have significant room for improvement with no model reaching near-ceiling performance.