Bonsai 27B
A 27B model pruned down to 3.9GB — small enough to live in your pocket
What it is
PrismML compresses Qwen3.6-27B to extremes: a ternary build ({−1, 0, +1}, 1.71 bits/weight, 5.9GB) keeping 94.6% of FP16 quality across 15 benchmarks, and a 1-bit build (1.125 bits/weight, 3.9GB) keeping 89.5% — running ~11 tok/s on an iPhone 17 Pro Max. Ships as GGUF for llama.cpp and MLX for Apple Silicon, with forked runtimes on GitHub.
Why it's interesting
The first credible 27B-class model that fits on a phone moves the local-AI frontier from 'gaming laptop' to 'thing already in your pocket' — fully offline, no telemetry, and Apache-2.0 end to end, inherited cleanly from its Apache base.
Use cases
- On-device assistants on phones with ~6GB free memory
- Offline laptop inference without a discrete GPU
- Extreme-quantization research with open recipes
Who it's for
Local-AI enthusiasts, mobile developers, quantization researchers
Setup
Easy. 1-bit build needs ~6GB free memory (iOS-capable); ternary wants ~8GB; runs via PrismML's llama.cpp and MLX forks
Limitations & cautions
The quality retention is uneven — vision tasks drop hardest (to ~60–65% of baseline), the 1-bit build's losses are real in the tails, and you're tied to PrismML's forked runtimes until upstream support lands.
Editorial takeaway
Colibri streams a 744B model off a hard drive; Bonsai shrinks a 27B into a phone. Two opposite bets, one thesis: the hardware you own is enough.