User ports KVarN to Bonsai runtime for 68% speedup and <10GB VRAM at 120K context

A user successfully ported the KVarN structured KV cache quantization method into PrismML's Bonsai runtime, enabling the Bonsai-Ternary-27B model to run efficiently on hardware with limited memory.

• The implementation resulted in a 68% increase in generation speed (73 vs 43 tok/s) and reduced VRAM usage by 3.3GB at 120K context.
• Total VRAM consumption dropped from approximately 13.1GB to 9.8GB, allowing the model to fit comfortably under 10GB.
• KVarN applies Walsh-Hadamard transform and Sinkhorn variance balancing before quantization to preserve quality better than flat quantization methods.
• The method is noted as more promising for retention-heavy use cases compared to TurboQuant, though it breaks DFlash compatibility.

This approach allows users running dense models at long context lengths to fit them into lower VRAM constraints without significant accuracy loss.