Written by Jakub Rusinowski · Last updated March 16, 2026
Model library → Mistral Small 4 → Mistral Small 4 119B-A6.5B
Unifies Mistral's reasoning, multimodal, and agentic-coding lines into one MoE checkpoint. 128 experts, 4 active per token (~6.5B active). At Q4_K_M it fits a single 24GB consumer GPU (RTX 4090/A10); FP8 and higher precision need 48GB+ workstation cards. 256K context, Apache 2.0.
Mistral Small 4 119B-A6.5B needs about 73 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
| Parameters | 119 Billion (~6.5B active) |
| Context window | 256,000 |
| Architecture | Mixture-of-Experts (128 experts, 4 active) |
| Provider | Mistral AI |
| Licence | Apache 2.0 |
| Specified at | Q4_K_M |
| System RAM | 32 GB |
| Record updated | 2026-03-16 |
Apache-2.0 — commercial use permitted. Commercial use permitted. No usage restrictions beyond attribution.
Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.
| Quant | Weights | VRAM needed | Est. speed | Fit on 24 GB |
|---|---|---|---|---|
| Q2_K | 39.1 GB | 39.9 GB | ~11 tok/s (est.) | Offloads to system RAM (slow) |
| Q3_K_M | 50.7 GB | 51.5 GB | ~9 tok/s (est.) | Offloads to system RAM (slow) |
| Q4_K_M | 71.8 GB | 72.6 GB | — | Won't fit |
| Q5_K_M | 84.3 GB | 85.1 GB | — | Won't fit |
| Q6_K | 97.6 GB | 98.4 GB | — | Won't fit |
| Q8_0 | 126.4 GB | 127.2 GB | — | Won't fit |
| F16 | 238.0 GB | 238.8 GB | — | Won't fit |
Want the memory numbers alone, at every quantization level and your own context length? Use the Mistral Small 4 119B-A6.5B VRAM calculator.
or compare on Vast.ai from $0.77/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
The cheapest catalogued GPU that runs Mistral Small 4 119B-A6.5B is the AMD Ryzen AI Max+ 395 (96 GB).
Install Ollama, then run:
ollama run mistral-small (community GGUF quants; check tag for 119B build)Weights on Hugging Face: mistralai/Mistral-Small-4-119B-2603.
Best for: reasoning, coding, multimodal, consumer gpu, agentic tasks.
← All Mistral Small 4 models | VRAM calculator | Check your own hardware