Audit finds AWQ, GPTQ, OWQ saliency metrics rely on shared input statistic

An independent researcher audited four widely used post-training quantization methods—AWQ, GPTQ, OWQ, and SpQR—to determine if their channel-importance scores accurately reflect weight sensitivity. By perturbing individual weight channels in Qwen2.5-0.5B, Pythia-410M, and SmolLM2-360M models at INT3 group-128 settings, the study measured real damage via forward-KL divergence.

• The four criteria showed high agreement, but this was largely mechanical due to sharing one input statistic rather than converging on truth.
• When that shared statistic is removed, three of the four methods barely track real sensitivity anymore.
• A gradient-based score remained robust and held up throughout the testing process.
• The author notes that end-to-end validation often masks poor importance picking because error-compensation steps cover for bad selections.

The researcher is seeking feedback on potential methodological holes and looking for an endorsement to submit the paper to arXiv (cs.LG).