1-bit Bonsai 27B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
The phone build. Binary {−1, +1} weights at ~1.125 bits/weight bring a full 27B multimodal model down to 3.9 GB — small enough to run on an iPhone 17 Pro at ~11 tok/s. PrismML reports it keeps more than 90% of the FP16 baseline (MATH 91.66, coding 81.88); tool-calling is the weak spot, dropping from ~80 to ~66. Ships as GGUF (llama.cpp / LM Studio) and MLX (Apple). Specs from launch coverage — verify on the Hugging Face model card.
1-bit Bonsai 27B needs about 5 GB of VRAM at 1-bit (binary, ~1.125 bpw) — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| 1-bit (binary, ~1.125 bpw) 1.13 bpw | 4.6 GB | ~119 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. 1-bit Bonsai 27B VRAM calculator →
Get 1-bit Bonsai 27B running
The cheapest catalogued GPU that runs 1-bit Bonsai 27B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run 1-bit Bonsai 27B
Install Ollama, then run:
ollama run bonsai-27bSpecifications
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
- Parameters
- 27 Billion
- Context window
- 262,144
- Architecture
- Qwen3.6-27B (multimodal), 1-bit binary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- 1-bit (binary, ~1.125 bpw)
- System RAM
- 8 GB
- Record updated
- 2026-07-15
Commercial use permitted. No usage restrictions beyond attribution.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
| Benchmark | Score | Provenance |
|---|---|---|
| MATH | 91.66 % | vendor-claimed · PrismML (launch) |
| Coding (HumanEval+ class) | 81.88 % | vendor-claimed · PrismML (launch) |
| Tool-calling (BFCL class) | 66 % | vendor-claimed · PrismML (launch) |
Can I run 1-bit Bonsai 27B on my GPU?
- 1-bit Bonsai 27B on AMD Radeon RX 9060 XT 8GB
- 1-bit Bonsai 27B on Intel Arc B570
- 1-bit Bonsai 27B on Intel Arc B580
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 3060 (12GB)
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 3070
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 3070 Ti
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 3080 (10GB)
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 4060
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 4070
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 4070 Super
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 4070 Ti
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 5060
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 5060 Ti 8GB
- 1-bit Bonsai 27B on NVIDIA GeForce RTX 5070
Other Bonsai 27B sizes
1-bit Bonsai 27B — frequently asked questions
How much VRAM does 1-bit Bonsai 27B need?
About 5 GB at 1-bit (binary, ~1.125 bpw) — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does 1-bit Bonsai 27B run on an RTX 4090 (24 GB)?
Yes. 1-bit Bonsai 27B needs about 5 GB at 1-bit (binary, ~1.125 bpw), inside a 24 GB card, at an estimated 119 tokens/sec.
How do I run 1-bit Bonsai 27B locally?
Install Ollama and run `ollama run bonsai-27b`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.
What other sizes does Bonsai 27B come in?
1-bit Bonsai 27B (5 GB), Ternary Bonsai 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.