media Hugging Face Forums · 2h ago · open_models

PIN v2 decouples model width from storage cost via weight tying

from English

PIN Architecture v2 introduces a method to pre-select a neural network's size and inference speed before training by using weight tying. This technique forces groups of cells to hold identical values, allowing the network to maintain its width while drastically reducing the number of stored weights.

Tying decouples network width from storage cost; a model with 256 units can fit within a 3,000-number budget that would normally allow only one hidden unit.
In tests against magnitude pruning, low-rank factorization, quantization, and distillation, tying won four of five bit budgets and was the only method functional below 100 KB.
At 128x compression, the tied model performs within 1.2 points of the uncompressed version.
The approach enables a shared coordinate system across models, allowing a base network and a linear map to predict variants for unseen task combinations without retraining.
A total of 43,000 numbers (base plus map) can emit any of 120 possible tasks, whereas conventional storage would require 14 million numbers.

This framework allows developers to specify storage budgets and inference bounds first, letting the model architecture follow those constraints rather than tuning until a fit is found.

Importance 1/3 Trust 1/3 Hugging Face Forums Inference efficiency Training methods

Related articles

arxiv arXiv cs.LG · 10d ago · 2 views

SkewAdam uses tiered optimizer state to reduce MoE training memory by 97%

Researchers introduce SkewAdam, an optimizer designed for mixture-of-experts (MoE) models that allocates different state types to distinct parameter populations to drastically reduce memory usage. By assigning float32 momentum and factored second moments to the backbone, factored second moments to experts, and exact second moments to the router, SkewAdam reduces optimizer state from 50.6 GB to 1.29 GB for a 6.78B-parameter model.

media Hugging Face Forums · 15d ago

OLMo-core ablation shows Engram indexing fixes narrow architecture gap

An ablation study in OLMo-core compared integrating Engram onto a standard Transformer versus a 3 GDN-layer + 1 attention-layer hybrid backbone. The analysis revealed that fixing signed overflow and padded token ID indexing failures substantially narrowed the performance gap between the two architectures.

media r/LocalLLaMA · 18d ago · 1 view

Flint compresses reasoning traces to reduce token usage while maintaining accuracy

A study demonstrates that section-aware compression of self-distilled reasoning traces allows Qwen3.5-4B and Gemma-4-12b models to match or exceed original performance using significantly fewer tokens. By preserving computation and verification spans while compressing narration, the method avoids the greedy decoding failures associated with flat compression.

media r/LocalLLaMA · 19d ago · 1 view

Flash-MSA accelerates million-token training with sparse attention kernels

Flash-MSA is a method designed to accelerate the training of models on sequences containing millions of tokens by utilizing sparse attention kernels.

lab NVIDIA Technical Blog · 21d ago

NVIDIA explains host offloading to reduce HBM bottlenecks in JAX LLM training

Large language model (LLM) training workloads increasingly hit GPU memory limits before compute is fully utilized. Model weights, gradients, optimizer states, communication buffers, and intermediate activations all compete for limited GPU high-bandwidth memory (HBM). As model size, sequence length, and batch size grow, HBM capacity often becomes the primary scaling bottleneck.