PALS adjusts per-layer sparsity for LLM pruning based on activation magnitudes

The authors propose PALS (Percentile-Aware Layerwise Sparsity), a one-shot pruning method that adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to ±5% around the target ratio. This approach addresses the limitation of methods like Wanda and SparseGPT, which apply uniform sparsity ratios regardless of layer importance.

• On LLaMA-2-7B at 50% sparsity, PALS achieves a WikiText-2 perplexity of 10.96, compared to 12.92 for uniform Wanda (mean over 9 runs, p < 0.001).
• The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none.
• Gradient-based allocation produces results worse than random, suggesting gradient magnitude does not predict the impact of discrete weight removal.

PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.