arxiv arXiv cs.AI · 2d ago · src: 3d ago · research

TokenProbe reduces CoT token usage by 76% while preserving reasoning quality

from English

Researchers present TokenProbe, a framework that addresses the high token consumption of Chain-of-Thought reasoning by exploiting the non-uniform value of tokens within a sequence. The method uses normalized log probability to distinguish core structural tokens from redundant filler, allowing for selective compression.

TokenProbe identifies core tokens carrying decisive reasoning content versus low-confidence exploratory filler using log probability signals.
It introduces an efficient GRPO objective that selectively compresses redundant tokens to improve the accuracy-token efficiency trade-off.
The approach reduces token usage by 76% compared to the baseline while maintaining reasoning quality.
Under matched reasoning-length budgets, TokenProbe outperforms strong flagship baselines like Gemini-3.1-Pro.

This work demonstrates that compressing low-value tokens yields Pareto improvements, enabling efficient reasoning without sacrificing performance.

Importance 2/3 arXiv cs.AI Inference efficiency Research paper

Read original

media Hugging Face Forums · 11d ago · 21 views

GlucoEdge: on-device glucose trend forecasting with INT8 quantization

Independent researcher Mohamed Menasy has published GlucoEdge, an engineering study detailing an end-to-end on-device machine learning pipeline for five-class, 15-minute glucose trend forecasting from continuous glucose monitor data. The work introduces a compact 1D CNN containing only 2,909 parameters and covers the full workflow from PyTorch to LiteRT conversion.

media Hugging Face Forums · 11d ago · 35 views

Literature review finds compression savings correlate weakly with measured energy

An independent researcher has published a literature review analyzing whether model compression techniques like quantization, pruning, and distillation actually reduce measured energy consumption or merely lower FLOPs.

lab Hugging Face Blog · 36d ago · 44 views

Multiverse Computing's QAH makes 4-bit GPT-OSS outperform its full-precision original

Multiverse Computing introduces Quantization-Aware Healing (QAH), a method that distills compressed, quantized large language models directly from their original pre-compression teachers. Applied to a GPT-OSS 120B model compressed to 60B parameters and quantized to MXFP4, the approach produces a model that beats its own full-precision bfloat16 version on 7 of 9 benchmarks.

arxiv arXiv cs.CL · 48d ago · 36 views

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

Researchers propose Reduced Matrix Multiplication (RMM), a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along contraction dimensions without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy-efficiency trade-off across language models ranging from 1B to 70B parameters.

lab NVIDIA Research · 50d ago · 51 views

DSTP mitigates visual token pruning failures in complex MLLM reasoning

Researchers identify Relevant Visual Information Shift (RVIS) during decoding as the primary cause for the failure of existing visual token pruning methods in Multimodal Large Language Models (MLLMs). To address this, they propose Decoding-stage Shift-aware Token Pruning (DSTP), a training-free framework that aligns visual tokens with shifting reasoning requirements.