Written by Jakub Rusinowski · Last updated September 6, 2026
Model library → Mistral Small 3.2 → Mistral Small 3.2 24B
23.6B dense with vision, 128K context, Apache 2.0. Mistral describes it explicitly as surgical: no architecture change from 3.1, just better instruction adherence, tool use and structured output, plus HumanEval Plus at 92.90%. 15.0 GB at Q4_K_M.
Mistral Small 3.2 24B needs about 15 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
| Parameters | 23.6 Billion |
| Context window | 131,072 |
| Architecture | Dense Transformer (vision) |
| Provider | Mistral AI |
| Licence | Apache 2.0 |
| Specified at | Q4_K_M |
| System RAM | 32 GB |
| Record updated | 2026-09-06 |
Apache-2.0 — commercial use permitted. Commercial use permitted. No usage restrictions beyond attribution.
Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.
| Quant | Weights | VRAM needed | Est. speed | Fit on 24 GB |
|---|---|---|---|---|
| Q2_K | 7.8 GB | 8.6 GB | ~74 tok/s (est.) | Fits comfortably |
| Q3_K_M | 10.1 GB | 10.9 GB | ~60 tok/s (est.) | Fits comfortably |
| Q4_K_M | 14.2 GB | 15.0 GB | ~45 tok/s (est.) | Fits comfortably |
| Q5_K_M | 16.7 GB | 17.5 GB | ~39 tok/s (est.) | Fits comfortably |
| Q6_K | 19.4 GB | 20.2 GB | ~35 tok/s (est.) | Fits comfortably |
| Q8_0 | 25.1 GB | 25.9 GB | ~4 tok/s (est.) | Offloads to system RAM (slow) |
| F16 | 47.2 GB | 48.0 GB | ~2 tok/s (est.) | Offloads to system RAM (slow) |
Want the memory numbers alone, at every quantization level and your own context length? Use the Mistral Small 3.2 24B VRAM calculator.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
The cheapest catalogued GPU that runs Mistral Small 3.2 24B is the AMD Radeon RX 9060 XT 16GB (16 GB).
Install Ollama, then run:
ollama run mistral-small:24bWeights on Hugging Face: mistralai/Mistral-Small-3.2-24B-Instruct-2506.
Quality scores as published by the model's authors or an independent evaluator — not throughput, and not measured by us.
| Benchmark | Score | Provenance |
|---|---|---|
| HumanEval Plus | 92.9 / 100 % | reported · https://docs.mistral.ai/models/mistral-small-3-2-25-06 |
| Arena Hard | 43.1 / 100 % | reported · https://venturebeat.com/ai/mistral-just-updated-its-open-source-small-model-from-3-1-to-3-2-heres-why |
Best for: general purpose, coding, function calling, multimodal.
← All Mistral Small 3.2 models | VRAM calculator | Check your own hardware