Rune 26B-A4B v3 — VRAM & local setup
Written by Jakub Rusinowski · Last updated
Surogate's 26B mixture-of-experts decision model (4B active) on Gemma 4 26B-A4B, with image input. Top downloadable model on the Decision Index 0.2.1 snapshot of 28 September 2026; served by surogate serve.
Rune 26B-A4B v3 needs about 16 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
Call Rune 26B-A4B v3
Served by surogate serve. This model does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text image
- Serve with
surogate serve- Endpoint
/v1/decisions
Setup for servers other than Ollama →
Not in the request builder: Surogate serves POST /v1/decisions, and the builder writes /v1/systemone requests.
Related guides: Confidence, thresholds and calibration · Decision models on images · Servers other than Ollama
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown. Image tokens are not modelled: each image adds to the prompt, so a request with images needs more memory than these rows.
| Quant | Memory | VRAM | Fit |
|---|---|---|---|
| BF16 Publisher build · 51.6 GB file | 54.1 GB | Offload | |
| Q4_K_M 4.83 bpw · modelled quant | 18.1 GB | Fits | |
| Q6_K 6.56 bpw · modelled quant | 23.7 GB | Tight | |
| Q8_0 8.50 bpw · modelled quant | 29.9 GB | Offload |
Published file size: BF16 51.6 GB. A download size from the model publisher — not a VRAM requirement.
The repository is named “-GGUF” but holds BF16 safetensors (11 shards, 51.6 GB), and it is gated: accept its terms on Hugging Face before downloading. The Decision Index served it with surogate serve (v1.5.3) at POST /v1/decisions, not /v1/systemone. The model card could not be read, so its context length, thinking mode and calibration controls are not listed here.
How it was scored
Two different suites on two different scales. Never compare the numbers across the two cards.
Decision Index 0.2.1 (snapshot 2026-09-28)
Decision Index 0.2.1Community Decision Index (not by the model's authors)- ECE (lower is better)
- 0.1196
- Median compute
- 120.5 ms on 1x NVIDIA RTX PRO 6000 (96 GB)
Community-maintained; not affiliated with the model authors.
Source ↗Decision models are not ranked on chat, creative or coding scores.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source. Still unconfirmed: questionTypes, activeParameters, contextWindow.
- Parameters
- 25.81B
- Context window
- Not published
- Architecture
- Fine-tune of google/gemma-4-26B-A4B-it (mixture-of-experts)
- Provider
- Surogate
- Licence
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Single consumer GPU
- Specified at
- Q4_K_M
- System RAM
- 34 GB
- Record updated
- 2026-10-13
Commercial use permitted. No usage restrictions beyond attribution.
Rune 26B-A4B v3 — frequently asked questions
What is Rune 26B-A4B v3?
Rune 26B-A4B v3 is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run Rune 26B-A4B v3 locally?
Served by surogate serve. This model does not run in Ollama.
How much memory does Rune 26B-A4B v3 need?
About 16 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is Rune 26B-A4B v3?
Its published score is on the community Decision Index 0.2.1, which its authors did not build. It is not a calibration guarantee: test on your own examples.