PrismML releases Bonsai 27B, a ternary Qwen3.6 model running on 10GB memory

PrismML has released Bonsai 27B, a new model that applies the Ternary/BitNet methodology to Qwen3.6 27B. This approach allows the model to run with near fp16 precision while consuming only approximately 10GB of memory on an M4 Pro chip.

• The model utilizes a ternary GGUF format, requiring a specific llama.cpp fork for inference.
• It supports a 256K context window and multi-modal input capabilities.
• Benchmarks indicate performance superior to traditional 2-3bit quantizations of Qwen3.6 27B.
• The release includes an MLX fork alongside the primary GGUF implementation.

This release makes running a high-utility 27B model on-device practical for laptops and desktops with limited VRAM, addressing previous constraints on local deployment.