OpenDecider-small — VRAM & /v1/systemone setup
Autor: Jakub Rusinowski · Ostatnia aktualizacja:
Model decyzyjny 4B: adapter LoRA na Qwen3-4B-Instruct-2507, destylowany z kalibrowanych modeli-nauczycieli. W pełnej precyzji mieści się na Macu z 16 GB; wersje GGUF działają w LM Studio i Ollama.
OpenDecider-small potrzebuje około 3 GB VRAM przy Q4_K_M — skwantyzowane wagi plus narzut frameworka, bez pamięci podręcznej KV. Na Apple Silicon ta wartość pochodzi z pamięci zunifikowanej.
Call OpenDecider-small
Served by opendecider serve (default port 8000).
- Question types
- choice noul (yes / no) score
- Inputs
- text
- Serve with
opendecider serve- Endpoint
/v1/systemone
pip install "opendecider[serve]"opendecider serve --model manjunathshiva/opendecider-smallhttp://localhost:8000/v1/systemone
Setup for servers other than Ollama →
Build a request for this model Decision models guide
Related guides: Confidence, thresholds and calibration · Servers other than Ollama
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.
| Kwant. | Pamięć | VRAM | Dopasowanie |
|---|---|---|---|
| BF16 (base + adapter) Publisher build · 8.19 GB file | 9.9 GB | Mieści się | |
| Q4_K_M 4.83 bpw · modelled quant | 4.2 GB | Mieści się | |
| Q6_K 6.56 bpw · modelled quant | 5 GB | Mieści się | |
| Q8_0 8.50 bpw · modelled quant | 6 GB | Mieści się |
Published file size: Q4_K_M 2.5 GB · Q8_0 4.28 GB · BF16 (base + adapter) 8.19 GB. A download size from the model publisher — not a VRAM requirement.
GGUF builds (Q8_0 4.28 GB, Q4_K_M 2.50 GB) load in Ollama and LM Studio. Ollama's own /v1/systemone builds a different prompt from the one this model was trained on, which costs about 9 points of accuracy on typed-decisions (0.583 vs 0.669, the author's figure): load the GGUF through the opendecider library, or use opendecider serve.
How it was scored
Each table is one suite. Numbers in different tables were measured on different questions and are not comparable, with each other or with the Decision Index.
Decision Index 0.2.1 (snapshot 2026-09-28)
Decision Index 0.2.1Community Decision Index (not by the model's authors)OpenDecider is not on the Decision Index 0.2.1 snapshot (28 September 2026), and its author has not published an Index score. Community-maintained; not affiliated with the model authors.
Source ↗| What was scored | Score | Source |
|---|---|---|
| Test split | 67.2% | Source ↗ |
Zero-shot: OpenDecider-small was not trained on this suite.
| What was scored | Score | ECE | Source |
|---|---|---|---|
| 200 decisions | 73.5% | 0.087 | Source ↗ |
Scored by the authors with the public Jev-vs-Laya harness (https://github.com/pavanjava/jev_and_laya_benchmarking). ECE is measured on this suite, not on typed-decisions.
Published latency
| Device | Time | Condition | Who measured it | Source |
|---|---|---|---|---|
| NVIDIA L40S | 37.6 ms | 1 question per call | Authors' own internal benchmark — not independently reproduced | Source ↗ |
| Apple M4 Max | 141 ms | 1 question per call | Authors' own internal benchmark — not independently reproduced | Source ↗ |
| Mac mini M4, 16 GB | 280 ms | 1 question per call | Authors' own internal benchmark — not independently reproduced | Source ↗ |
As published, on the device and under the condition shown. A decision model answers in one forward pass, so this is time per decision, not tokens per second.
Decision models are not ranked on chat, creative or coding scores.
Specyfikacja
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parametry
- 4.02B
- Okno kontekstu
- Not published
- Architektura
- Fine-tune of Qwen/Qwen3-4B-Instruct-2507
- Dostawca
- Manjunath Janardhan
- Licencja
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Określono przy
- Q4_K_M
- RAM systemowy
- 8 GB
- Rekord zaktualizowany
- 2026-10-13
Commercial use permitted. No usage restrictions beyond attribution.
Inne rozmiary OpenDecider
OpenDecider-small — najczęściej zadawane pytania
What is OpenDecider-small?
OpenDecider-small is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run OpenDecider-small locally?
Served by opendecider serve (default port 8000). Start it with: `pip install "opendecider[serve]" && opendecider serve --model manjunathshiva/opendecider-small`.
How much memory does OpenDecider-small need?
About 3 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is OpenDecider-small?
It has no community Decision Index score. Every score on this page comes from its authors, on the suites named beside each table; none is an independent result.