Olmo 3.1 32B Instruct — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
Ai2's fully open 32B chat model: weights, training data, code and checkpoints are all published. A dense transformer with three sliding-window layers for every full-attention layer and a 65,536-token context. This build answers directly; the Think build reasons first. Apache-2.0.
Olmo 3.1 32B Instruct needs about 20 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 12.8 GB | ~57 tok/s | Fits | |
| Q3_K_M 3.41 bpw | 16 GB | ~46 tok/s | Fits | |
| Q4_K_M 4.83 bpw | 21.7 GB | ~34 tok/s | Tight | |
| Q5_K_M 5.67 bpw | 25.1 GB | ~4 tok/s | Offload | |
| Q6_K 6.56 bpw | 28.7 GB | ~4 tok/s | Offload | |
| Q8_0 8.50 bpw | 36.5 GB | ~3 tok/s | Offload | |
| F16 16.00 bpw | 66.7 GB | — | Too big |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Olmo 3.1 32B Instruct VRAM calculator →
Get Olmo 3.1 32B Instruct running
The cheapest catalogued GPU that runs Olmo 3.1 32B Instruct is the AMD Radeon RX 7900 XTX (24 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Olmo 3.1 32B Instruct
Install Ollama, then run:
ollama run olmo-3.1:32b-instructSpecifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 32 Billion
- Context window
- 65,536
- Architecture
- Dense transformer, 3:1 sliding-window (4,096) to full attention
- Provider
- Ai2 (Allen Institute for AI)
- Licence
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Single consumer GPU
- Specified at
- Q4_K_M
- System RAM
- 32 GB
- Record updated
- 2026-11-11
- Runs with
- Ollama: supported source
Commercial use permitted. No usage restrictions beyond attribution.
See also Olmo 3.1 32B Think
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Other Olmo 3 sizes
Olmo 3.1 32B Instruct — frequently asked questions
How much VRAM does Olmo 3.1 32B Instruct need?
About 20 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Olmo 3.1 32B Instruct run on an RTX 4090 (24 GB)?
Yes. Olmo 3.1 32B Instruct needs about 20 GB at Q4_K_M, inside a 24 GB card, at an estimated 34 tokens/sec.
How do I run Olmo 3.1 32B Instruct locally?
Install Ollama and run `ollama run olmo-3.1:32b-instruct`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.
What other sizes does Olmo 3 come in?
Olmo 3 7B (5 GB), Olmo 3 32B (20 GB), Olmo 3.1 32B Instruct (20 GB), Olmo 3.1 32B Think (20 GB), Olmo Hybrid 7B (5 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.