The Bonsai-27B and Ternary-Bonsai-27B models are seeing increased integration into the mainline llama.cpp repository, with binary Q1_0 support now available out of the box across CPU, Metal, CUDA, and Vulkan backends. Ternary support is currently in a transitional phase, with CPU and Metal backends merged while CUDA and Vulkan implementations remain in review or pending.
- Binary Q1_0 format is fully supported in upstream llama.cpp; MLX support remains pending via a separate pull request.
- Three ternary GGUF variants exist: the demo's group-128 Q2_0 (fork-only), the official group-64 Q2_0 (mainline CPU/Metal), and the planned PQ2_0 format.
- Bonsai-27B retains 89.5% of full-precision quality, while Ternary-Bonsai-27B retains 94.6%, with both noting agentic coding is not yet a primary focus.
- The ternary model's ~7.2 GB footprint exceeds iOS memory budgets, requiring the 1-bit MLX variant for mobile deployment.
These updates clarify the current compatibility landscape for users running these extreme compression models on various hardware backends.