Ternary Bonsai 2 9B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
An 8.95B-parameter language model plus a 0.46B vision tower, rebuilt from Qwen3.5-9B with ternary weights: 32 blocks, of which 8 use full attention and 24 linear attention. The PTQ1_0 file is 2.0 GB, PQ2_0 2.42 GB and Q2_0 2.56 GB — the same weights in three layouts. 262,144-token context, image input, tool calling, with an MTP drafter built into a separate file. Thinking mode can spend 3,000+ tokens even on short prompts, so the card advises allowing at least 8,192 output tokens with it on.
Ternary Bonsai 2 9B needs about 3 GB of VRAM at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| PTQ1_0 ternary 1.78 bpw | 3.1 GB | ~198 tok/s | Fits | |
| PQ2_0 ternary 2.16 bpw | 3.5 GB | ~178 tok/s | Fits | |
| Q2_0 ternary 2.28 bpw | 3.6 GB | ~172 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Ternary Bonsai 2 9B VRAM calculator →
Get Ternary Bonsai 2 9B running
The cheapest catalogued GPU that runs Ternary Bonsai 2 9B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Ternary Bonsai 2 9B
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer.
Specifications
Corroborated — Two or more independent sources agree on these figures, but the model card itself was not retrieved. Treat the numbers as good rather than confirmed. Still unconfirmed: parameter count (model card only; the Hub's GGUF metadata reports 1.47B for this packed file).
- Parameters
- 8.95 Billion (+0.46B vision tower)
- Context window
- 262,144
- Architecture
- Qwen3.5-9B hybrid attention (24 linear + 8 full), ternary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- PTQ1_0 ternary
- Vision projector
- +0.6 GB when you send images (not included above)
- System RAM
- 8 GB
- Record updated
- 2026-10-15
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- Ships only in PrismML's ternary packings; there is no Q4_K_M, Q8_0 or FP16 GGUF to choose instead.
- Upstream llama.cpp loads only the Q2_0 file; PTQ1_0, PQ2_0 and the drafters need the PrismML fork. The released mlx-lm (0.32.0) cannot load the MLX build yet; mlx-vlm 0.7.8 can.
- The vision projector is a separate file (0.62 GB at Q8_0).
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Ternary Bonsai 2 9B on my GPU?
Other Ternary Bonsai 2 sizes
Ternary Bonsai 2 9B — frequently asked questions
How much VRAM does Ternary Bonsai 2 9B need?
About 3 GB at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Ternary Bonsai 2 9B run on an RTX 4090 (24 GB)?
Yes. Ternary Bonsai 2 9B needs about 3 GB at PTQ1_0 ternary, inside a 24 GB card, at an estimated 198 tokens/sec.
How do I run Ternary Bonsai 2 9B locally?
No Ollama library entry. The packed ternary formats need PrismML's llama.cpp fork (or upstream llama.cpp for the Q2_0 file only); on Apple devices use the MLX build with mlx-vlm 0.7.8 or newer. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does Ternary Bonsai 2 come in?
Ternary Bonsai 2 2B (1 GB), Ternary Bonsai 2 4B (2 GB), Ternary Bonsai 2 9B (3 GB), Ternary Bonsai 2 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.