Open Jev reproductions beyond Ollama

Ollama ships three decision models. The community has published about 70 open reproductions of Jev, and several score far higher on the Decision Index — Winnow-12B (50.0) fits a 16 GB card at Q8_0, and AutoJev-27B (56.4) comes within 1.5 points of Jev (57.9) but needs about 49 GiB for its BF16 weights. Almost all of them expose the same POST /v1/systemone request shape, so code you wrote against Ollama keeps working — you change the URL and port.

When to leave Ollama's three models

Stay with nimble / tev1 if they're accurate enough on 50–100 of your own labelled examples. Move on when:

• your questions are harder than routing — multi-step judgments, knowledge questions, long policies — where the Index gap between Nimble (39.6) and the leaders (50–57) shows up;
• you need more than 26 options in a choice (Ollama's limit; Decider and Jev go to 255);
• you need non-English input (Laya's multilingual router covers 100+ languages; most others are English-first);
• you need images in the state (AutoJev, Intern-Decision and Winnow accept images; Ollama's endpoint doesn't).

The shortlist

Only models with open weights, a permissive licence, and a documented local runtime made the cut. Decision Index 0.2.1 snapshot of 28 September 2026 (chance-corrected; 0 = random, 100 = perfect).

² Laya is an encoder built for fast routing and guardrails in many languages, not a general reasoner — its Index score reflects that. It's on the list because nothing else covers 100+ languages locally.

Left off on purpose: the top-ranked Surogate Rune 26B-A4B (57.4) because its Hugging Face repo listed no GGUF file when we checked on 30 September; the JPT family because it is CC-BY-NC-4.0 (non-commercial); and every "inference technique" entry on the Index, because those are decoding methods applied to stock models rather than weights you download. We'll add Rune once its files are published.

Fit on your hardware: set your GPU or unified memory in the hardware analyzer and this page shows whether each model fits it. Check your hardware →

Pattern 1 — GGUF + llama.cpp-based server (Winnow)

The most hardware-flexible path. Winnow's own server is built on llama.cpp and exposes /v1/systemone and a normal /v1/chat/completions from the same loaded model — one download gives you a decision model and a chat/vision model.

Tested setups, per its README: Linux with an NVIDIA GPU (RTX 5070 Ti, CUDA 12.8+), Apple Silicon with 24 GB or more of unified memory, and an NVIDIA Docker container. The Q8_0 build was run with a 64K context on a 16 GB RTX 5070 Ti.

# bash — Linux (NVIDIA) or macOS (Apple Silicon)
git clone https://github.com/EldanRing/winnow-inference
cd winnow-inference
python3 scripts/setup.py                             # downloads the model (~12.9 GB)
python3 scripts/serve.py --profile 5070ti-64k        # NVIDIA
# python3 scripts/serve.py --profile apple-silicon   # Mac

It listens on http://127.0.0.1:8091. Your Ollama request works unchanged apart from the URL:

curl http://127.0.0.1:8091/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"state":"The store accepts returns within 30 days. This item was bought 12 days ago.",
       "questions":{"eligible":{"type":"noul","instructions":"Is the item eligible for return?"}}}'

Check the repo's quickstart for whether model is required and for the current profile names — this project is two weeks old and moves quickly.

Pattern 2 — Python package with a built-in server (Decider, JevK5, AutoJev)

These ship their own inference code and a small HTTP server that speaks TypeSafe's format. You need a working PyTorch install with GPU support first.

Decider — the most downloaded open decision model (over 255,000 downloads of the 2B in its first two weeks). Four sizes share one interface.

# Python 3.11+; needs torch, transformers>=5 and flash-linear-attention
# (it runs without flash-linear-attention, but several times slower)
from decider.infer import Decider   # the decider/ package ships inside the model repo

d = Decider("Mapika/decider-2b")
d.decide(
    "My card was charged twice for the same purchase.",
    [{"question": "Which department should handle this?", "options": ["billing", "technical support", "sales"]},
     {"question": "Does this need a refund action?", "options": ["no", "yes"]}],
)

For an HTTP endpoint, run decider.serve with DECIDER_MODEL pointing at the downloaded model folder; the official typesafe-sdk works against it unchanged with TYPESAFE_BASE_URL set to that server. Decider allows 2–255 options per question, and abstain_below=t makes it return None instead of an answer when confidence is under t.

JevK5 — one of the best-calibrated 4B models on the Index (expected calibration error 0.027). English only; inputs over 16,384 tokens are refused rather than cut.

# bash — needs a CUDA GPU with ~9 GB free for BF16
jevk5-serve --model alibiserikbay/JevK5 --port 8090

No NVIDIA card? The alibiserikbay/JevK5-GGUF builds run under llama.cpp on NVIDIA, AMD, Intel and Apple GPUs or the CPU. Per its card, the Q8_0 file matches the full-precision answer on 229 of 231 public JevBench items, and Q4_K_M on 221.

AutoJev-27B — the strongest open model you can actually download today. Its card asks for a GPU with room for about 49 GiB of BF16 weights plus runtime overhead — in practice a 64 GB-class or larger workstation card.

# bash — Python 3.12+ and uv
git clone https://github.com/denis-pplx/autojev.git
cd autojev
uv sync --frozen --python 3.12
uv run hf download denis-pplx/autojev-27b --local-dir checkpoints/selected
AUTOJEV_CHECKPOINT=checkpoints/selected uv run autojev-serve

It serves a browser playground at http://localhost:8000 and POST /v1/systemone with optional base64 images. Set AUTOJEV_API_KEY to turn on authentication — do that before exposing it to a network.

Pattern 3 — A tiny encoder for routing in any language (Laya)

pip install laya            # extras: laya[serve], laya[mcp], laya[langchain], laya[onnx]

from laya import Router

router = Router()   # downloads a checkpoint on first use
result = router.predict(
    "Hi, we were billed twice for March. Please refund the duplicate today or we will cancel our plan.",
    {"department": {"type": "choice", "instructions": "Which department should handle this?",
                    "criteria": {"billing": "invoices, payments, refunds",
                                 "technical": "bugs, outages, system errors",
                                 "other": "everything else"}}},
)
print(result["answers"]["department"]["choice"], result["routing"]["model"])

The router detects the script and language and sends non-English text to the multilingual checkpoint automatically. It's the fastest thing on this page and the least capable at reasoning — use it as a first-pass router, not a judge.

Which one for your hardware?

This table is guidance from the published file sizes and each project's tested setups. Your analyzer result is the verdict for your exact machine. If the model you need doesn't fit what you have, renting a GPU by the hour is often cheaper than buying for an occasional batch job — compare in the cost calculator.

Train your own

Two of these projects publish their full recipe:

• Together AI's Tev1 — data recipe and training code at github.com/togethercomputer/tev1, with a Together blog post on training your own classifier for about $17.
• Decider — code, data registry and training scripts at github.com/Mapika/decider.

A decision model fine-tuned on a few thousand of your own labelled decisions usually beats a bigger general one on your task. Our fine-tuning guides cover the basics of LoRA on a single GPU.

Frequently asked questions

Verified against: Decision Index 0.2.1 data (generated 28 Sep 2026); Hugging Face model cards and GitHub READMEs read 30 Sep 2026.

Keep going

• Decision models (Jev-style System One): run them locally→
• Run Jev-style decision models locally with Ollama (Nimble, Tev1)→
• Decision-model confidence, thresholds and calibration — use the probabilities safely→
• Ollama /v1/systemone errors and fixes→
• Decision request builder→