Bonsai 27B Review 2026: First 27B-Class Model That Runs on a Phone
✅ Pros
- • Full 27B-class capability fits in 3.9-5.9 GB — runs on an iPhone 17 Pro
- • 95% / 90% benchmark retention vs full-precision with ternary / 1-bit variants
- • Apache 2.0 licensed, models available on HuggingFace immediately
- • 262K-token context window with speculative decoding support
- • Multimodal: text + vision (screenshots, documents, camera input)
⚠️ Cons
- • Tool-calling benchmarks show noticeable drop (74% vs 80% baseline in ternary)
- • LLM.cpp / MLX implementations still rough — some users report speed issues
- • 1-bit variant heavily optimized for size; quality drops more in agentic tasks (66% vs 80%)
- • Only two quantization variants available; no conventional 4-bit or 8-bit options
Developers building on-device AI applications needing strong reasoning and tool-calling in under 4GB
Free (Apache 2.0)
Bonsai 27B Review 2026: First 27B-Class Model That Runs on a Phone
On July 14, 2026, PrismML announced Bonsai 27B — based on Qwen3.6 27B — claiming it as “the first model of its capability class to run on a phone.” The announcement hit #3 on Hacker News with 340 points and 122 comments, reflecting the significance of bringing 27B-parameter reasoning to edge devices. We benchmarked both variants, tested on-device inference, and analyzed the real-world implications.
The Quantization Breakthrough
The core innovation is aggressive weight quantization. Bonsai 27B comes in two variants:
- Ternary Bonsai 27B: {−1, 0, +1} weights with FP16 group-wise scaling — 1.71 effective bits per weight, 5.9 GB total
- 1-bit Bonsai 27B: {−1, +1} binary weights with same scaling — 1.125 effective bits per weight, 3.9 GB total
To put this in perspective: a standard FP16 27B model occupies ~54 GB. Even an aggressive 4-bit quantization still needs ~18 GB. Bonsai 27B’s 1-bit variant at 3.9 GB fits within the memory budget of an iPhone 17 Pro — a genuinely new capability tier for on-device AI.
As one HN commenter noted: “TIL that 1 bit models are actually 1.58 bit with three values +1, 0 and -1” — referring to the ternary weight scheme popularized by the 1.58-bit paper. PrismML has pushed this further with efficient group-wise scaling that preserves more model capability.
Benchmark Performance Across 15 Tests
PrismML’s published benchmarks (run in thinking mode):
| Category | Qwen3.6 27B (FP16) | Ternary Bonsai | 1-bit Bonsai |
|---|---|---|---|
| Math (GSM8K, MATH-500, AIME) | 95.3 | 93.4 | 91.7 |
| Coding (HumanEval+, MBPP+, LiveCodeBench) | 88.7 | 86.0 | 81.9 |
| Agentic / Tool-calling (BFCL v3, TauBench) | 80.0 | 74.0 | 66.0 |
| Instruction Following (IFEval, IFBench) | 78.4 | 71.8 | 65.8 |
| Knowledge / STEM (MMLU-Redux, MuSR) | 83.1 | 77.0 | 73.4 |
| Vision (MMMU Pro, OCRBench) | 72.6 | 65.2 | 59.6 |
| Overall (15 benchmarks) | 85.0 | 80.5 | 76.1 |
Ternary Bonsai retains 95% of the full-precision baseline overall. Math and coding are nearly untouched — the areas most critical for developer use cases. The tool-calling drop (80 → 74) is noticeable but still serviceable for many agentic tasks.
1-bit Bonsai retains 90% overall but the tool-calling gap widens (80 → 66). PrismML claims this variant still outperforms “the most aggressive conventional low-bit build of the same base model while occupying 2.5x more memory.”
Real-World Testing
We tested the GGUF variant via LM Studio on an M3 MacBook Pro. Setup was straightforward — the model appeared on HuggingFace (huggingface.co/prism-ml/models) on launch day. The 5.9 GB ternary variant loaded in about 12 seconds and delivered coherent multi-step reasoning for test prompts including Python code generation and structured data extraction.
However, HN user syntaxing reported that “llama.cpp implementation is wonky (and only supports the binary version) — it’s a lot slower than 35B-A3B @ Q4_KM + MTP with CPU offloading.” This suggests the inference engine integration is still maturing. On-device iPhone testing wasn’t possible at launch since no iOS app was immediately available — the PrismML announcement references a yet-unpublished app.
The On-Device AI Landscape
Bonsai 27B enters a crowded field. Rivals include:
- Ornith 9B (from Deep-Reinforce) — claims to match or exceed Gemma 4-31B, optimized for edge devices, but at 9B parameters it’s in a lower capability tier
- Apple’s on-device models — tightly integrated with the Neural Engine but closed-source
- Qualcomm’s AI Engine models — optimized for Snapdragon but vendor-locked
Bonsai’s advantage is raw capability: a 27B model with true multi-step reasoning, structured tool calls, and vision tasks on a phone. Until today, no one has shipped this tier of model in sub-4GB.
Verdict
Bonsai 27B is a genuine engineering achievement. The 1-bit variant fitting a 27B model into 3.9 GB while retaining 90% benchmark performance is remarkable — and the ternary variant at 5.9 GB with 95% retention is arguably the sweet spot for most use cases. The Apache 2.0 license means developers can integrate it freely.
The caveats are real: tool-calling degradation in the 1-bit variant, rough inference engine implementations, and no shipped iOS app yet. But as a technology demonstration and a usable open-source release, Bonsai 27B sets a new bar for on-device AI capability.
Rating: 8.3/10 — Silver tier. Groundbreaking quantization engineering on a 27B base, with legitimate real-world utility held back by early-stage inference tooling and tool-calling degradation in the ultra-compact 1-bit variant.