kev 4B — VRAM & /v1/systemone setup
Written by Jakub Rusinowski · Last updated
A 4B decision model: a LoRA and a pointer head on Qwen3.5-4B-Base, Apache-2.0. llama.cpp's server answers it natively at /v1/systemone from the official ggml-org GGUF (3.0 GB at Q4_K_M).
kev 4B needs about 4 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
Call kev 4B
Served by llama-server (llama.cpp) (default port 9931). This model does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text
- Serve with
llama-server (llama.cpp)- Endpoint
/v1/systemone
llama-server -hf ggml-org/Kev-4B-GGUF:Q8_0http://localhost:9931/v1/systemone
Setup for servers other than Ollama →
Build a request for this model Decision models guide
Related guides: Servers other than Ollama · Confidence, thresholds and calibration
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · Decider 12B Vision · Decider 31B v2 (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M · mammoth-1 27B · elephant-1 9B · Strands Decider 2B (hobson-v21) · kev 9B · OpenJev 27B
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.
| Quant | Memory | VRAM | Fit |
|---|---|---|---|
| BF16 Publisher build · 8.43 GB file | 10.2 GB | Fits | |
| Q4_K_M 4.83 bpw · modelled quant | 4.8 GB | Fits | |
| Q6_K 6.56 bpw · modelled quant | 5.6 GB | Fits | |
| Q8_0 8.50 bpw · modelled quant | 6.3 GB | Fits |
Published file size: Q4_K_M 3.03 GB · Q8_0 4.48 GB · BF16 8.43 GB. A download size from the model publisher — not a VRAM requirement.
Validated to 8,192 tokens of state; the server accepts up to 65,536. The Decision Index row is 'Kev 4B r10'.
How it was scored
Two different suites on two different scales. Never compare the numbers across the two cards.
Decision Index 0.4.1 (snapshot 2026-10-22)
Decision Index 0.4.1Community Decision Index (not by the model's authors)- ECE (lower is better)
- 0.032
Community-maintained; not affiliated with the model authors.
Source ↗Decision models are not ranked on chat, creative or coding scores.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 4.66B
- Context window
- 8K validated (65,536 served)
- Architecture
- Fine-tune of Qwen/Qwen3.5-4B-Base
- Provider
- Jared Palmer (community)
- Licence
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- Q4_K_M
- System RAM
- 8 GB
- Record updated
- 2026-10-25
- Runs with
- llama.cpp: supported source
Commercial use permitted. No usage restrictions beyond attribution.
Other kev sizes
kev 4B — frequently asked questions
What is kev 4B?
kev 4B is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run kev 4B locally?
Served by llama-server (llama.cpp) (default port 9931). This model does not run in Ollama. Start it with: `llama-server -hf ggml-org/Kev-4B-GGUF:Q8_0`.
How much memory does kev 4B need?
About 4 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is kev 4B?
Its published score is on the community Decision Index 0.4.1, which its authors did not build. It is not a calibration guarantee: test on your own examples.