mammoth-1 27B — VRAM & local setup
Written by Jakub Rusinowski · Last updated
Base Labs' 27B decision model, a full fine-tune of Qwen3.5-27B (text only), MIT. Third of 228 on the Jev Decision Index 0.4.1 (22 Oct 2026); served by Base Labs' CUDA runtime, not llama.cpp or Ollama.
mammoth-1 27B needs about 17 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
Call mammoth-1 27B
Served by Base Labs runtime (OneJev + serve.patch). This model does not run in Ollama.
- Question types
- choice noul (yes / no) score
- Inputs
- text
- Serve with
Base Labs runtime (OneJev + serve.patch)
Setup for servers other than Ollama →
Not in the request builder: Base Labs' runtime is a Python engine (engine.decide), and its HTTP route is not documented on the card.
Related guides: Servers other than Ollama · Confidence, thresholds and calibration
Other decision models: Nimble 9B · Tev1 4B · Tev1 0.8B · Winnow 12B · Winnow E4B · Decider 2B · Decider 4B · Decider 35B-A3B (NVFP4) · Decider 12B Vision · Decider 31B v2 (NVFP4) · JevK5 4B · Intern-Decision 4B · AutoJev 27B · Laya · Clef 27B · Clef-Flash 9B · Rune 26B-A4B v3 · OpenDecider-nano · OpenDecider-small · OpenDecider-small MLX 4-bit · Bosun v3.1 0.6B · Bosun v3.1 1.7B · d1-3B · d1-omni-600M · THX-01 322M · elephant-1 9B · Strands Decider 2B (hobson-v21) · kev 4B · kev 9B · OpenJev 27B
Hardware fit
Weights plus overhead plus the KV cache at a 8,192-token prompt, on NVIDIA RTX 4090 (24 GB). A publisher build is sized from its file; the other rows are modelled at a standard quant. Decision requests are short, so no long-context figure is shown.
| Quant | Memory | VRAM | Fit |
|---|---|---|---|
| BF16 Publisher build · 54.71 GB file | 57.3 GB | Too big | |
| Q4_K_M 4.83 bpw · modelled quant | 19.1 GB | Fits | |
| Q6_K 6.56 bpw · modelled quant | 25 GB | Offload | |
| Q8_0 8.50 bpw · modelled quant | 31.6 GB | Offload |
Published file size: BF16 54.71 GB. A download size from the model publisher — not a VRAM requirement.
The prithivMLmods GGUF is a plain llama.cpp conversion with chat-style instructions; it does not claim the decision readout, so treat it as a chat model.
How it was scored
Two different suites on two different scales. Never compare the numbers across the two cards.
Decision Index 0.4.1 (snapshot 2026-10-22)
Decision Index 0.4.1Community Decision Index (not by the model's authors)- ECE (lower is better)
- 0.0408
Community-maintained; not affiliated with the model authors.
Source ↗Decision models are not ranked on chat, creative or coding scores.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 27.36B
- Context window
- 16K (16,384)
- Architecture
- Fine-tune of Qwen/Qwen3.5-27B
- Provider
- Base Labs (Baseten)
- Licence
- mit
- Availability
- Weights downloadable
- Practical deployment
- Single consumer GPU
- Specified at
- Q4_K_M
- System RAM
- 36 GB
- Record updated
- 2026-10-25
Commercial use permitted. No usage restrictions beyond attribution.
Other mammoth-1 / elephant-1 sizes
mammoth-1 27B — frequently asked questions
What is mammoth-1 27B?
mammoth-1 27B is a decision model: you send it a state and typed questions (choice, yes/no/unknown, or a score) and it returns one answer per question with a probability for every option. It is not a chat model.
How do I run mammoth-1 27B locally?
Served by Base Labs runtime (OneJev + serve.patch). This model does not run in Ollama.
How much memory does mammoth-1 27B need?
About 17 GB for the weights plus overhead at Q4_K_M, before the prompt's KV cache. Decision prompts are short, so the cache stays small.
How accurate is mammoth-1 27B?
Its published score is on the community Decision Index 0.4.1, which its authors did not build. It is not a calibration guarantee: test on your own examples.