elephant-1 9B — VRAM & local setup
Written by Jakub Rusinowski · Last updated
Base Labs' 9B decision model, a full fine-tune of Qwen3.5-9B with its vision encoder kept frozen, so it reads images; MIT. Served by Base Labs' CUDA runtime.
elephant-1 9B needs about 6 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
Call elephant-1 9B
Served by Base Labs runtime (OneJev + serve.patch). This model does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text image
- Serve with
Base Labs runtime (OneJev + serve.patch)
Setup for servers other than Ollama →
Not in the request builder: Base Labs' runtime is a Python engine (engine.decide), and its HTTP route is not documented on the card.
Related guides: Servers other than Ollama · Confidence, thresholds and calibration
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · Decider 12B Vision · Decider 31B v2 (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M · mammoth-1 27B · Strands Decider 2B (hobson-v21) · kev 4B · kev 9B · OpenJev 27B
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown. Image tokens are not modelled: each image adds to the prompt, so a request with images needs more memory than these rows.
| Quant | Memory | VRAM | Fit |
|---|---|---|---|
| BF16 Publisher build · 18.82 GB file | 20.8 GB | Fits | |
| Q4_K_M 4.83 bpw · modelled quant | 7.7 GB | Fits | |
| Q6_K 6.56 bpw · modelled quant | 9.7 GB | Fits | |
| Q8_0 8.50 bpw · modelled quant | 12 GB | Fits |
Published file size: BF16 18.82 GB. A download size from the model publisher — not a VRAM requirement.
The prithivMLmods GGUF is a plain llama.cpp conversion with chat-style instructions; it does not claim the decision readout, so treat it as a chat model.
How it was scored
Two different suites on two different scales. Never compare the numbers across the two cards.
Decision Index 0.4.1 (snapshot 2026-10-22)
Decision Index 0.4.1Community Decision Index (not by the model's authors)- ECE (lower is better)
- 0.0578
Community-maintained; not affiliated with the model authors.
Source ↗Decision models are not ranked on chat, creative or coding scores.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 9.41B
- Context window
- 16K (16,384)
- Architecture
- Fine-tune of Qwen/Qwen3.5-9B
- Provider
- Base Labs (Baseten)
- Licence
- mit
- Availability
- Weights downloadable
- Practical deployment
- Laptop
- Specified at
- Q4_K_M
- System RAM
- 14 GB
- Record updated
- 2026-10-25
Commercial use permitted. No usage restrictions beyond attribution.
Other mammoth-1 / elephant-1 sizes
elephant-1 9B — frequently asked questions
What is elephant-1 9B?
elephant-1 9B is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run elephant-1 9B locally?
Served by Base Labs runtime (OneJev + serve.patch). This model does not run in Ollama.
How much memory does elephant-1 9B need?
About 6 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is elephant-1 9B?
Its published score is on the community Decision Index 0.4.1, which its authors did not build. It is not a calibration guarantee: test on your own examples.