Developer eramax has released an open-source pipeline and llama.cpp patch that compresses the large GLM-5.2 Mixture of Experts model to fit on a single RTX 3090 with 24 GB VRAM.
The approach uses real-activation whitening and a shared basis for gate and up matrices, reducing mean expert size to ~1.4 MiB while maintaining a mean offline cosine similarity of 0.959. The compressed codec tree reduces storage requirements from hundreds of gigabytes in the raw GGUF format to single-digit gigabytes. Despite successful per-layer compression and loading, end-to-end text generation remains broken due to stacked errors across the 75 layers.
The work serves as a research handoff for further development on codec design and calibration rather than a finished product.