Ternary Bonsai 2 27B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
PrismML's ternary rebuild of Qwen3.8-27B: 26.9B parameters in the language model and embeddings plus a 0.46B vision tower, 64 blocks with about one in four using full attention. The PTQ1_0 file is 5.95 GB and PQ2_0 7.21 GB, against about 54 GB at FP16. PrismML reports it retains 98.2% of the FP16 model's average across 14 thinking-mode benchmarks. Stock llama.cpp will NOT run these files: it rejects PTQ1_0 and PQ2_0, and per the card it loads the Q2_0 build without warning and produces garbage. Use PrismML's llama.cpp or MLX fork.
Ternary Bonsai 2 27B needs about 7 GB of VRAM at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| PTQ1_0 ternary 1.78 bpw | 7.3 GB | ~95 tok/s | Fits | |
| PQ2_0 ternary 2.16 bpw | 8.5 GB | ~82 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Ternary Bonsai 2 27B VRAM calculator →
Get Ternary Bonsai 2 27B running
The cheapest catalogued GPU that runs Ternary Bonsai 2 27B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Ternary Bonsai 2 27B
No Ollama library entry, and stock llama.cpp cannot run these files. Use PrismML's llama.cpp fork (CUDA, Metal, CPU) or its MLX fork on Apple Silicon.
Specifications
Corroborated — Two or more independent sources agree on these figures, but the model card itself was not retrieved. Treat the numbers as good rather than confirmed. Still unconfirmed: parameter count (model card and GGUF metadata agree on 26.9B for the language model; no independent source).
- Parameters
- 26.9 Billion (+0.46B vision tower)
- Context window
- 262,144
- Architecture
- Qwen3.8-27B hybrid attention (~3 linear : 1 full), ternary weights
- Provider
- PrismML
- Licence
- Apache 2.0
- Availability
- Weights downloadable
- Practical deployment
- Laptop
- Specified at
- PTQ1_0 ternary
- Vision projector
- +0.6 GB when you send images (not included above)
- System RAM
- 16 GB
- Record updated
- 2026-10-15
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- Ships only in PrismML's ternary packings; there is no Q4_K_M, Q8_0 or FP16 GGUF to choose instead.
- Stock llama.cpp, Ollama and LM Studio cannot run it; the PrismML forks are required.
- The card's speed table was measured on an earlier build and is marked pending re-measurement.
- The vision projector is a separate file (0.63 GB at Q8_0).
See also Ternary Bonsai 27B
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Ternary Bonsai 2 27B on my GPU?
Other Ternary Bonsai 2 sizes
Ternary Bonsai 2 27B — frequently asked questions
How much VRAM does Ternary Bonsai 2 27B need?
About 7 GB at PTQ1_0 ternary — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Ternary Bonsai 2 27B run on an RTX 4090 (24 GB)?
Yes. Ternary Bonsai 2 27B needs about 7 GB at PTQ1_0 ternary, inside a 24 GB card, at an estimated 95 tokens/sec.
How do I run Ternary Bonsai 2 27B locally?
No Ollama library entry, and stock llama.cpp cannot run these files. Use PrismML's llama.cpp fork (CUDA, Metal, CPU) or its MLX fork on Apple Silicon. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does Ternary Bonsai 2 come in?
Ternary Bonsai 2 2B (1 GB), Ternary Bonsai 2 4B (2 GB), Ternary Bonsai 2 9B (3 GB), Ternary Bonsai 2 27B (7 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.