The paper presents BiSCo-LLM, a codebook-free framework designed for extreme low-bit large language model weight compression. It addresses the limitations of existing methods by eliminating explicit codebooks and index lookups while maintaining representation capacity.
- Local weight chunks are mapped onto a unit hypersphere and binarized into compact spherical codes, using a bit-packed sign stream as the main payload.
- A residual BSQ stage encodes reconstruction errors to provide an explicit rate-distortion path without stored codebooks.
- Category-wise recovery distillation is applied after replacing Transformer module categories to reduce mismatch between local weight reconstruction and model behavior.
- An 8-bit protected-channel path stabilizes sensitive channels and is counted separately from the BSQ payload.
The framework offers a storage budget that includes binary codes, neural decoders, protected-channel payloads, LoRA adapters, and metadata.