media r/LocalLLaMA · 1h ago · open_models

Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original

from English

The article introduces Quantization-Aware Healing, a method for creating a compressed 4-bit model that achieves performance superior to its full-precision original.

This approach demonstrates that specific quantization techniques can enhance model capabilities beyond the baseline unquantized version.

Importance 2/3 r/LocalLLaMA Inference efficiency Open weights

Read original

media r/LocalLLaMA · 20h ago · 1 view

Facebook releases MobileMoE, a family of on-device MoE language models

Meta has released MobileMoE, a collection of on-device Mixture-of-Experts (MoE) language models designed to optimize the quality-efficiency trade-off for mobile devices. The family includes three model scales with sub-billion active parameters and weight footprints under 3 GB in INT4 format.

arxiv arXiv cs.LG · 2d ago · 12 views

Quantization-Aware Healing recovers 4-bit LLMs faster than QAT

The authors introduce Quantization-Aware Healing (QAH), a pipeline that distills 4-bit students directly from uncompressed models to recover performance lost during structural compression and quantization. Applied to GPT-OSS 120B, this method produces Hypernova-60B, which matches or beats the bfloat16 source on 7 of 9 benchmarks while using roughly 4 times less weight memory.

arxiv arXiv cs.CL · 2d ago · 11 views

Quantization-Aware Healing recovers 4-bit LLMs faster than QAT

The authors introduce Quantization-Aware Healing (QAH), a method to recover reasoning and coding capabilities in structurally compressed, 4-bit large language models. Unlike standard quantization-aware training which re-fits the compressed model, QAH distills the 4-bit student directly from the original uncompressed model.

media Hugging Face Forums · 3d ago · 5 views

Xyntetik Runner trains LoRA directly through quantized GGUF weights

Joakimpalm-Zen has updated Xyntetik Runner to train LoRA adapters directly using the same quantized GGUF weights it serves for inference, eliminating the need for a separate FP16 training copy or runtime.

media MarkTechPost · 5d ago · 5 views

Liquid AI releases LFM2.5-DSpark draft models for up to 3.18x faster decoding

Liquid AI has released DSpark draft model checkpoints for three models in its LFM2.5 family: LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B. These drafters add a speculative decoding path to existing target models, allowing a roughly 300M-parameter draft to propose a block of nine candidate tokens that the target model verifies in a single forward pass.