Written by Jakub Rusinowski · Last updated September 6, 2026
Model library → Magistral Small → Magistral Small 24B
24B dense reasoning model with image understanding, 128K context, Apache 2.0. Emits long reasoning traces before answering, so budget for the extra tokens. 15.3 GB at Q4_K_M leaves comfortable headroom on a 24 GB card — a genuine local reasoning model rather than a distilled approximation of one.
Magistral Small 24B needs about 15 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
| Parameters | 24 Billion |
| Context window | 131,072 |
| Architecture | Dense Transformer (reasoning-tuned, vision) |
| Provider | Mistral AI |
| Licence | Apache 2.0 |
| Specified at | Q4_K_M |
| System RAM | 32 GB |
| Record updated | 2026-09-06 |
Apache-2.0 — commercial use permitted. Commercial use permitted. No usage restrictions beyond attribution.
Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.
| Quant | Weights | VRAM needed | Est. speed | Fit on 24 GB |
|---|---|---|---|---|
| Q2_K | 7.9 GB | 8.7 GB | ~73 tok/s (est.) | Fits comfortably |
| Q3_K_M | 10.2 GB | 11.0 GB | ~59 tok/s (est.) | Fits comfortably |
| Q4_K_M | 14.5 GB | 15.3 GB | ~44 tok/s (est.) | Fits comfortably |
| Q5_K_M | 17.0 GB | 17.8 GB | ~39 tok/s (est.) | Fits comfortably |
| Q6_K | 19.7 GB | 20.5 GB | ~34 tok/s (est.) | Fits comfortably |
| Q8_0 | 25.5 GB | 26.3 GB | ~4 tok/s (est.) | Offloads to system RAM (slow) |
| F16 | 48.0 GB | 48.8 GB | ~2 tok/s (est.) | Offloads to system RAM (slow) |
Want the memory numbers alone, at every quantization level and your own context length? Use the Magistral Small 24B VRAM calculator.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
The cheapest catalogued GPU that runs Magistral Small 24B is the AMD Radeon RX 9060 XT 16GB (16 GB).
Install Ollama, then run:
ollama run magistral:24bWeights on Hugging Face: mistralai/Magistral-Small-2509.
Best for: reasoning, math, coding, multimodal.
← All Magistral Small models | VRAM calculator | Check your own hardware