lab NVIDIA Research · 15h ago · research

DSTP mitigates visual token pruning failures in complex MLLM reasoning

from English

Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.

DSTP is a training-free add-on that corrects RVIS during the decoding stage.
It significantly reduces performance degradation of pruning methods in complex reasoning tasks.
The method yields consistent performance gains across visual understanding benchmarks.
DSTP demonstrates generalizability and efficiency with minimal computational overhead across diverse state-of-the-art architectures.

This approach allows existing pruning techniques to maintain reliability when handling the vast number of visual tokens required for complex visual reasoning.

Importance 1/3 Trust 3/3 NVIDIA Research Inference efficiency Research paper

Read original

media Hugging Face Forums · 7d ago · 1 view

Open-source Decay-Gated O(N) Causal Linear Attention with fused Triton kernel

A developer has open-sourced a Decay-Gated O(N) Causal Linear Attention architecture featuring fused Triton/CUDA state accumulation kernels designed to bypass quadratic MHA bottlenecks.

media Hugging Face Forums · 13d ago · 5 views

Jason Van Pham documents Gemma 4 multi-turn greedy degeneration under letter-level constraints

Jason Van Pham and co-engineer Grok (xAI) published a diagnostic report detailing a near-vanilla multi-turn failure mode in the Hydrodynamic Swarm harness when using a custom Gemma 4 instruct path. The study identifies that combining greedy decoding with long token budgets and letter-level constraints triggers classical text degeneration, including repetition and orthographic thrash.

arxiv arXiv cs.CL · 15d ago · 2 views

PIVOT shares prefix scans across query groups to accelerate token-level sparse attention

PIVOT (Proxy Indexing Via One full-prefix Traversal) is a training-free, drop-in replacement for the DeepSeek Sparse Attention (DSA) indexer that reduces computational redundancy by sharing one prefix scan across a group of nearby queries. Instead of scoring every preceding token for each query individually, PIVOT aggregates the group into a single proxy query to obtain a candidate set, from which top-k tokens are selected for each query.

arxiv arXiv cs.CL · 26d ago · 3 views

ShortOPD recovers pruned LLMs with short-to-long on-policy distillation

The authors propose ShortOPD, a short-to-long on-policy distillation schedule designed to recover the generative capabilities of structured-pruned large language models that collapse during free-form generation. By detecting teacher-confirmed repetitive suffixes and treating the surviving prefix as the effective rollout length, the method allocates training budget more efficiently than standard approaches.

arxiv arXiv cs.CL · 27d ago · 1 view

Adaptive filtering corrects structural-role bias in LLM KV cache eviction

The article diagnoses how attention-based KV cache eviction methods like H2O disproportionately retain noise on schema-dense inputs, such as nested JSON, where structural tokens carry significantly more energy than content tokens. This bias causes exact-match accuracy to collapse from 88% to 0% at a 5% budget due to the over-retention of KEY tokens relative to VALUE tokens.