OpenDecider — Manjunath Janardhan 的本地 AI 模型
作者： Jakub Rusinowski · 最后更新：
从经过校准的教师模型蒸馏而来的开源决策模型：400M 编码器（nano）和基于 Qwen3-4B 的 4B LoRA（small），通过 pip 安装，由兼容 /v1/systemone 的 opendecider serve 提供服务。更大的 30B 与 80B MoE 版本需要 NVIDIA GPU，未在此列出。
变体
OpenDecider 最小的变体在 Q4_K_M 下约需 1 GB 显存——量化权重加框架开销，不含 KV 缓存。
| 模型 | Q4 下显存 | 显存 | 上下文 | 运行 |
|---|---|---|---|---|
| OpenDecider-nano → 0.4B | ~1 GB | Not published | opendecider serve :8000 | |
| OpenDecider-small → 4B | ~3.3 GB | Not published | opendecider serve :8000 | |
| OpenDecider-small MLX 4-bit → 4B | ~3.2 GB | Not published | opendecider (MLX) |
显存为 Q4_K_M 下的量化权重加开销，与 GPU 与显存检测器使用同一引擎计算。
如何在本地运行 OpenDecider
安装 Ollama，然后拉取标签。
Served by opendecider serve (default port 8000). This model does not run in Ollama.
在上方选择一个尺寸，查看它自己的显存、速度估算和安装命令。
许可证
Commercial use permitted. No usage restrictions beyond attribution.
适用于: OpenDecider-nano, OpenDecider-small, OpenDecider-small MLX 4-bit推荐 GPU
目录中能在本地运行 OpenDecider（至少 1 GB 显存）的最便宜 GPU 是 Intel Arc B570 (10 GB).
OpenDecider — 常见问题
How much VRAM does OpenDecider need?
OpenDecider needs about 1 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: OpenDecider-nano (1 GB, Q4_K_M); OpenDecider-small (3 GB, Q4_K_M); OpenDecider-small MLX 4-bit (3 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run OpenDecider on an RTX 4090 (24 GB)?
Yes — OpenDecider runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
What quantization should I use for OpenDecider?
Q4_K_M is the best balance of quality and VRAM for OpenDecider in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
How do I run OpenDecider locally?
OpenDecider is a decision model: it is called over an HTTP endpoint (/v1/systemone), not chatted with. Where a variant is available in Ollama 0.35 or newer, pull it with `ollama pull` and send requests to the local server; the others ship their own server. Each variant page shows the exact commands for that model.