Bosun v3.1 0.6B — VRAM & /v1/systemone setup
作者： Jakub Rusinowski · 最后更新：
Hanno Labs 最小的决策模型：基于 Qwen3-0.6B 的 rank-16 LoRA，带 256 个决策 token，每个问题最多 255 个候选项。由兼容 Jev 的服务器通过 /v1/systemone 提供服务。
Bosun v3.1 0.6B 在 Q4_K_M 下约需 1 GB 显存——量化权重加框架开销，不含 KV 缓存。在 Apple Silicon 上，这部分来自统一内存。
Call Bosun v3.1 0.6B
Served by jev-compatible-server (default port 8000). This model does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text
- Max candidates per question
- 255
- Serve with
jev-compatible-server- Endpoint
/v1/systemone- Languages
- English
uvx --from 'jev-compatible-server[transformers]' jev-compatible-server --model bosun-v3.1-0.6bhttp://localhost:8000/v1/systemone
Setup for servers other than Ollama →
Build a request for this model Decision models guide
Related guides: Confidence, thresholds and calibration · Servers other than Ollama
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.
| 量化 | 显存 | 显存 | 适配 |
|---|---|---|---|
| BF16 (base + adapter) Publisher build · 1.54 GB file | 2.9 GB | 可运行 | |
| Q4_K_M 4.83 bpw · modelled quant | 1.7 GB | 可运行 | |
| Q6_K 6.56 bpw · modelled quant | 2 GB | 可运行 | |
| Q8_0 8.50 bpw · modelled quant | 2 GB | 可运行 |
Published file size: Q4_K_M 0.4 GB · Q8_0 0.64 GB · BF16 (base + adapter) 1.54 GB. A download size from the model publisher — not a VRAM requirement.
How it was scored
Each table is one suite. Numbers in different tables were measured on different questions and are not comparable, with each other or with the Decision Index.
Decision Index 0.2.1 (snapshot 2026-09-28)
Decision Index 0.2.1Community Decision Index (not by the model's authors)- ECE (lower is better)
- 0.1419
- Median compute
- 38.7 ms on 1x NVIDIA RTX PRO 6000 (96 GB)
Community-maintained; not affiliated with the model authors.
Source ↗| What was scored | Score | ECE | NLL | Coverage | Source |
|---|---|---|---|---|---|
| Applied suite (22,700 rows) | 83.2% | 0.0588 | 0.621 | 100% | Source ↗ |
| All rows, including the 1,200-row reasoning track (23,900 rows) | 81.18% | 0.0659 | 0.6491 | 100% | Source ↗ |
The authors' caveat: these are held-out rows from task families represented in Bosun's training data, not evidence on unseen task families.
On the separate reasoning track, Jev leads both Bosun models (the authors’ statement).
Decision models are not ranked on chat, creative or coding scores.
规格
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- 参数量
- 0.75B
- 上下文窗口
- Not published
- 架构
- Fine-tune of Qwen/Qwen3-0.6B
- 提供商
- Hanno Labs
- 许可证
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- 规格量化
- Q4_K_M
- 系统内存
- 8 GB
- 记录更新于
- 2026-10-13
Commercial use permitted. No usage restrictions beyond attribution.
Bosun 的其他尺寸
Bosun v3.1 0.6B — 常见问题
What is Bosun v3.1 0.6B?
Bosun v3.1 0.6B is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run Bosun v3.1 0.6B locally?
Served by jev-compatible-server (default port 8000). This model does not run in Ollama. Start it with: `uvx --from 'jev-compatible-server[transformers]' jev-compatible-server --model bosun-v3.1-0.6b`.
How much memory does Bosun v3.1 0.6B need?
About 1 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is Bosun v3.1 0.6B?
It has separate published scores on different suites — the community Decision Index 0.2.1 and the benchmarks listed on this page. They are not comparable with each other, and none is a calibration guarantee.