OpenDecider-small MLX 4-bit — VRAM & Python setup
作者： Jakub Rusinowski · 最后更新：
面向 Apple Silicon（MLX）量化为 4 位的 OpenDecider-small：下载 2.3 GB，运行约占 2.6 GB 内存，在 typed-decisions 上比全精度低约 2 分。
OpenDecider-small MLX 4-bit 在 Q4_K_M 下约需 3 GB 显存——量化权重加框架开销，不含 KV 缓存。在 Apple Silicon 上，这部分来自统一内存。
Call OpenDecider-small MLX 4-bit
A Python library: you load it inside your own program. It has no HTTP server and does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text
- Serve with
opendecider (MLX)
pip install "opendecider[mlx]"Not in the request builder: this build is documented for the Python library only.
Related guides: Confidence, thresholds and calibration
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.
| 量化 | 显存 | 显存 | 适配 |
|---|---|---|---|
| MLX 4-bit Publisher build · 2.26 GB file | 4 GB | 可运行 | |
| Q4_K_M 4.83 bpw · modelled quant | 4.1 GB | 可运行 | |
| Q6_K 6.56 bpw · modelled quant | 5 GB | 可运行 | |
| Q8_0 8.50 bpw · modelled quant | 6 GB | 可运行 |
Published file size: MLX 4-bit 2.26 GB. A download size from the model publisher — not a VRAM requirement.
Apple Silicon only. About 2 points below the full-precision model on typed-decisions (0.651 vs 0.672). Serving this build over HTTP is not documented, so it is listed as a Python library.
How it was scored
Each table is one suite. Numbers in different tables were measured on different questions and are not comparable, with each other or with the Decision Index.
Decision Index 0.2.1 (snapshot 2026-09-28)
Decision Index 0.2.1Community Decision Index (not by the model's authors)OpenDecider is not on the Decision Index 0.2.1 snapshot (28 September 2026), and its author has not published an Index score. Community-maintained; not affiliated with the model authors.
Source ↗| What was scored | Score | Source |
|---|---|---|
| Test split | 65.1% | Source ↗ |
Zero-shot. The full-precision model scores 0.672 here.
| What was scored | Score | Source |
|---|---|---|
| 200 decisions | 74.5% | Source ↗ |
Scored by the authors with the public Jev-vs-Laya harness (https://github.com/pavanjava/jev_and_laya_benchmarking). ECE is measured on this suite, not on typed-decisions.
Published latency
| Device | Time | Condition | Who measured it | Source |
|---|---|---|---|---|
| Apple M4 Max, 64 GB | 65 ms | 1 short question | Authors' own internal benchmark — not independently reproduced | Source ↗ |
| Apple M4 Max, 64 GB | 143 ms | median over the benchmark questions | Authors' own internal benchmark — not independently reproduced | Source ↗ |
As published, on the device and under the condition shown. A decision model answers in one forward pass, so this is time per decision, not tokens per second.
Decision models are not ranked on chat, creative or coding scores.
规格
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- 参数量
- 4.02B
- 上下文窗口
- Not published
- 架构
- Fine-tune of Qwen/Qwen3-4B-Instruct-2507
- 提供商
- Manjunath Janardhan
- 许可证
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- 规格量化
- Q4_K_M
- 系统内存
- 8 GB
- 记录更新于
- 2026-10-13
- Runs with
- MLX: supported source
Commercial use permitted. No usage restrictions beyond attribution.
OpenDecider 的其他尺寸
OpenDecider-small MLX 4-bit — 常见问题
What is OpenDecider-small MLX 4-bit?
OpenDecider-small MLX 4-bit is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run OpenDecider-small MLX 4-bit locally?
A Python library: you load it inside your own program. It has no HTTP server and does not run in Ollama. Install: `pip install "opendecider[mlx]"`.
How much memory does OpenDecider-small MLX 4-bit need?
About 3 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is OpenDecider-small MLX 4-bit?
It has no community Decision Index score. Every score on this page comes from its authors, on the suites named beside each table; none is an independent result.