Sensitivity-Aware Thresholding and Token Routing for Activation Sparsification in Large Language Models

Researchers propose Sensitivity-Aware Thresholding for Sparsity (SATS) and a lightweight token routing framework to optimize inference in Large Language Models. SATS replaces percentile-based calibration with a local MLP output sensitivity proxy to determine layerwise gate thresholds, while the routing framework dynamically selects computation paths per token.

• SATS uses a sensitivity-aware selection rule instead of activation percentiles for threshold calibration.
• Token routing dynamically chooses between base and modified paths on a per-token basis.
• Evaluation on multiple open-weight LLMs shows SATS improves over threshold-based sparsification baselines at matched actual sparsity.
• Token routing yields a more favorable quality-throughput trade-off than static activation modification baselines.

The authors conclude that improved threshold calibration and token routing can enhance the quality-throughput trade-off in LLMs.