Ternary Bonsai 2 4B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
A 4.21B-parameter language model plus a 0.33B vision tower, rebuilt from Qwen3.5-4B with ternary weights. The PTQ1_0 file is 0.95 GB. 262,144-token context, image input, tool calling, and two optional speculative-decoding drafters (MTP and DFlash2) that need PrismML's llama.cpp fork. Upstream llama.cpp runs only the Q2_0 file.
Ternary Bonsai 2 4B needs about 2 GB of VRAM at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| PTQ1_0 ternary 1.78 bpw | 2 GB | ~273 tok/s | Fits | |
| PQ2_0 ternary 2.16 bpw | 2.2 GB | ~255 tok/s | Fits | |
| Q2_0 ternary 2.28 bpw | 2.3 GB | ~249 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Ternary Bonsai 2 4B VRAM calculator →
Get Ternary Bonsai 2 4B running
The cheapest catalogued GPU that runs Ternary Bonsai 2 4B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Ternary Bonsai 2 4B
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer.
Specifications
Corroborated — Two or more independent sources agree on these figures, but the model card itself was not retrieved. Treat the numbers as good rather than confirmed. Still unconfirmed: parameter count (model card only; the GGUF metadata under-counts packed ternary weights).
- Parameters
- 4.21 Billion (+0.33B vision tower)
- Context window
- 262,144
- Architecture
- Qwen3.5-4B hybrid attention (3 linear : 1 full), ternary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- PTQ1_0 ternary
- Vision projector
- +0.4 GB when you send images (not included above)
- System RAM
- 8 GB
- Record updated
- 2026-10-15
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- Ships only in PrismML's ternary packings; there is no Q4_K_M, Q8_0 or FP16 GGUF to choose instead.
- Upstream llama.cpp loads only the Q2_0 file; PTQ1_0, PQ2_0 and both drafters need the PrismML fork.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Ternary Bonsai 2 4B on my GPU?
Other Ternary Bonsai 2 sizes
Ternary Bonsai 2 4B — frequently asked questions
How much VRAM does Ternary Bonsai 2 4B need?
About 2 GB at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Ternary Bonsai 2 4B run on an RTX 4090 (24 GB)?
Yes. Ternary Bonsai 2 4B needs about 2 GB at PTQ1_0 ternary, inside a 24 GB card, at an estimated 273 tokens/sec.
How do I run Ternary Bonsai 2 4B locally?
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does Ternary Bonsai 2 come in?
Ternary Bonsai 2 2B (1 GB), Ternary Bonsai 2 4B (2 GB), Ternary Bonsai 2 9B (3 GB), Ternary Bonsai 2 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.