Ternary Bonsai 27B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
The laptop build and recommended pick when you have the memory. Ternary {−1, 0, +1} weights at ~1.71 bits/weight land at 5.9 GB while keeping more than 95% of the FP16 baseline. PrismML reports MATH-500 99.20, AIME 2025 90.84, HumanEval+ 93.90 and LiveCodeBench 82.8, with tool-use and agentic ability largely intact (~74). Multimodal, 262K context, Apache 2.0. Ships as GGUF and MLX; also hosted on Together AI. Specs from launch coverage — verify on the Hugging Face model card.
Ternary Bonsai 27B needs about 7 GB of VRAM at Ternary (1.58-bit, ~1.71 bpw) — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Ternary (1.58-bit, ~1.71 bpw) 1.71 bpw | 6.6 GB | ~91 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Ternary Bonsai 27B VRAM calculator →
Get Ternary Bonsai 27B running
The cheapest catalogued GPU that runs Ternary Bonsai 27B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Ternary Bonsai 27B
Install Ollama, then run:
ollama run bonsai-27bSpecifications
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
- Parameters
- 27 Billion
- Context window
- 262,144
- Architecture
- Qwen3.6-27B (multimodal), ternary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Practical deployment
- Laptop
- Specified at
- Ternary (1.58-bit, ~1.71 bpw)
- System RAM
- 16 GB
- Record updated
- 2026-07-15
Commercial use permitted. No usage restrictions beyond attribution.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
| Benchmark | Score | Provenance |
|---|---|---|
| MATH-500 | 99.2 % | vendor-claimed · PrismML (launch) |
| AIME 2025 | 90.84 % | vendor-claimed · PrismML (launch) |
| HumanEval+ | 93.9 % | vendor-claimed · PrismML (launch) |
| LiveCodeBench | 82.8 % | vendor-claimed · PrismML (launch) |
| Tool-use (agentic) | 74.01 % | vendor-claimed · PrismML (launch) |
| Vision | 65.19 % | vendor-claimed · PrismML (launch) |
Can I run Ternary Bonsai 27B on my GPU?
- Ternary Bonsai 27B on AMD Radeon RX 9060 XT 8GB
- Ternary Bonsai 27B on Intel Arc B570
- Ternary Bonsai 27B on Intel Arc B580
- Ternary Bonsai 27B on NVIDIA GeForce RTX 3060 (12GB)
- Ternary Bonsai 27B on NVIDIA GeForce RTX 3070
- Ternary Bonsai 27B on NVIDIA GeForce RTX 3070 Ti
- Ternary Bonsai 27B on NVIDIA GeForce RTX 3080 (10GB)
- Ternary Bonsai 27B on NVIDIA GeForce RTX 4060
- Ternary Bonsai 27B on NVIDIA GeForce RTX 4070
- Ternary Bonsai 27B on NVIDIA GeForce RTX 4070 Super
- Ternary Bonsai 27B on NVIDIA GeForce RTX 4070 Ti
- Ternary Bonsai 27B on NVIDIA GeForce RTX 5060
- Ternary Bonsai 27B on NVIDIA GeForce RTX 5060 Ti 8GB
- Ternary Bonsai 27B on NVIDIA GeForce RTX 5070
Other Bonsai 27B sizes
Ternary Bonsai 27B — frequently asked questions
How much VRAM does Ternary Bonsai 27B need?
About 7 GB at Ternary (1.58-bit, ~1.71 bpw) — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Ternary Bonsai 27B run on an RTX 4090 (24 GB)?
Yes. Ternary Bonsai 27B needs about 7 GB at Ternary (1.58-bit, ~1.71 bpw), inside a 24 GB card, at an estimated 91 tokens/sec.
How do I run Ternary Bonsai 27B locally?
Install Ollama and run `ollama run bonsai-27b`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.
What other sizes does Bonsai 27B come in?
1-bit Bonsai 27B (5 GB), Ternary Bonsai 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.