Ternary Bonsai 2 2B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
The smallest Bonsai 2: a 1.88B-parameter language model plus a 0.33B vision tower, rebuilt from Qwen3.5-2B with ternary weights. The densest packing (PTQ1_0) is a 0.43 GB file, small enough for any phone. 262,144-token context, image input, tool calling. Upstream llama.cpp runs only the larger Q2_0 file; PTQ1_0 and PQ2_0 need PrismML's fork, and on Apple devices the MLX build loads in mlx-vlm 0.7.8 or newer.
Ternary Bonsai 2 2B needs about 1 GB of VRAM at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| PTQ1_0 ternary 1.78 bpw | 1.3 GB | ~349 tok/s | Fits | |
| PQ2_0 ternary 2.16 bpw | 1.4 GB | ~335 tok/s | Fits | |
| Q2_0 ternary 2.28 bpw | 1.4 GB | ~331 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Ternary Bonsai 2 2B VRAM calculator →
Get Ternary Bonsai 2 2B running
The cheapest catalogued GPU that runs Ternary Bonsai 2 2B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Ternary Bonsai 2 2B
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer.
Specifications
Corroborated — Two or more independent sources agree on these figures, but the model card itself was not retrieved. Treat the numbers as good rather than confirmed. Still unconfirmed: parameter count (model card only; the GGUF metadata under-counts packed ternary weights).
- Parameters
- 1.88 Billion (+0.33B vision tower)
- Context window
- 262,144
- Architecture
- Qwen3.5-2B hybrid attention (3 linear : 1 full), ternary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- PTQ1_0 ternary
- Vision projector
- +0.4 GB when you send images (not included above)
- System RAM
- 8 GB
- Record updated
- 2026-10-15
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- Ships only in PrismML's ternary packings; there is no Q4_K_M, Q8_0 or FP16 GGUF to choose instead.
- Upstream llama.cpp loads only the Q2_0 file; PTQ1_0 and PQ2_0 need the PrismML fork.
- The vision projector is a separate file (0.37 GB at Q8_0); the card advises the Q8_0 projector because the BF16 one can crash llama-server under concurrent image requests.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Ternary Bonsai 2 2B on my GPU?
Other Ternary Bonsai 2 sizes
Ternary Bonsai 2 2B — frequently asked questions
How much VRAM does Ternary Bonsai 2 2B need?
About 1 GB at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Ternary Bonsai 2 2B run on an RTX 4090 (24 GB)?
Yes. Ternary Bonsai 2 2B needs about 1 GB at PTQ1_0 ternary, inside a 24 GB card, at an estimated 349 tokens/sec.
How do I run Ternary Bonsai 2 2B locally?
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does Ternary Bonsai 2 come in?
Ternary Bonsai 2 2B (1 GB), Ternary Bonsai 2 4B (2 GB), Ternary Bonsai 2 9B (3 GB), Ternary Bonsai 2 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.