PALS adjusts per-layer sparsity for LLM pruning using activation percentiles

The authors propose PALS (Percentile-Aware Layerwise Sparsity), a one-shot pruning method that adjusts per-layer sparsity based on the 99th percentile of activation magnitudes, bounded to ±5% around the target ratio. This approach addresses the limitation of existing methods like Wanda and SparseGPT, which apply uniform sparsity ratios across all layers.

• On LLaMA-2-7B at 50% sparsity, PALS achieves a WikiText-2 perplexity of 10.96, compared to 12.92 for uniform Wanda (mean over 9 runs, p < 0.001).
• The benefit is architecture-dependent: LLaMA-3-8B shows marginal gains and Mistral-7B shows none.
• Gradient-based allocation produces results worse than random, suggesting gradient magnitude does not predict the impact of discrete weight removal.
• PALS adds negligible cost to the pruning pipeline and requires no fine-tuning.