media Hugging Face Forums · 4d ago · open_models

Literature review finds compression savings correlate weakly with measured energy

from English

An independent researcher has published a literature review analyzing whether model compression techniques like quantization, pruning, and distillation actually reduce measured energy consumption or merely lower FLOPs.

The review draws on a corpus of 53 works from 2019 to 2026.
Software energy estimators and hardware counters (RAPL, NVML) disagree by margins comparable to the savings claimed by compression methods.
FLOPs correlate weakly with measured energy once memory bandwidth, batch size, and backend kernel realization are accounted for.
Compression rankings on one hardware platform do not reliably transfer to another.

The author shares the review and associated data tables on Zenodo and GitHub, while seeking a cs.LG endorsement to post the work on arXiv.

Importance 1/3 Trust 1/3 Hugging Face Forums Inference efficiency Research paper

Read original

media Hugging Face Forums · 3d ago · 17 views

GlucoEdge: on-device glucose trend forecasting with INT8 quantization

Independent researcher Mohamed Menasy has published GlucoEdge, an engineering study detailing an end-to-end on-device machine learning pipeline for five-class, 15-minute glucose trend forecasting from continuous glucose monitor data. The work introduces a compact 1D CNN containing only 2,909 parameters and covers the full workflow from PyTorch to LiteRT conversion.

lab Hugging Face Blog · 29d ago · 39 views

Multiverse Computing's QAH makes 4-bit GPT-OSS outperform its full-precision original

Multiverse Computing introduces Quantization-Aware Healing (QAH), a method that distills compressed, quantized large language models directly from their original pre-compression teachers. Applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, the approach produces a model that beats its own full-precision bfloat16 version on 7 of 9 benchmarks.

arxiv arXiv cs.CL · 40d ago · 31 views

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

Researchers propose Reduced Matrix Multiplication (RMM), a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along contraction dimensions without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy-efficiency trade-off across language models ranging from 1B to 70B parameters.

lab NVIDIA Research · 43d ago · 47 views

DSTP mitigates visual token pruning failures in complex MLLM reasoning

Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.

media Hugging Face Forums · 50d ago · 29 views

Open-source Decay-Gated O(N) Causal Linear Attention with fused Triton kernel

A developer has open-sourced a Decay-Gated O(N) Causal Linear Attention architecture featuring fused Triton/CUDA state accumulation kernels designed to bypass quadratic MHA bottlenecks.