SmolLM3 3B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
3.08B dense, pretrained on 11.2T tokens across web, code, math and reasoning data. Native English, French, Spanish, German, Italian and Portuguese with additional Arabic, Chinese and Russian training. Runs on effectively anything — 2.7 GB at Q4_K_M — and unlike most small models the entire training pipeline is reproducible.
SmolLM3 3B needs about 3 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 1.8 GB | ~242 tok/s | Fits | |
| Q3_K_M 3.41 bpw | 2.1 GB | ~220 tok/s | Fits | |
| Q4_K_M 4.83 bpw | 2.7 GB | ~190 tok/s | Fits | |
| Q5_K_M 5.67 bpw | 3 GB | ~176 tok/s | Fits | |
| Q6_K 6.56 bpw | 3.3 GB | ~163 tok/s | Fits | |
| Q8_0 8.50 bpw | 4.1 GB | ~140 tok/s | Fits | |
| F16 16.00 bpw | 7 GB | ~91 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. SmolLM3 3B VRAM calculator →
Get SmolLM3 3B running
The cheapest catalogued GPU that runs SmolLM3 3B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run SmolLM3 3B
Install Ollama, then run:
ollama run smollm3:3bSpecifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 3.08 Billion
- Context window
- 131,072
- Architecture
- Dense Transformer
- Provider
- Hugging Face
- Licence
- Apache 2.0
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- Q4_K_M
- System RAM
- 8 GB
- Record updated
- 2026-09-06
Commercial use permitted. No usage restrictions beyond attribution.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run SmolLM3 3B on my GPU?
SmolLM3 3B — frequently asked questions
How much VRAM does SmolLM3 3B need?
About 3 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does SmolLM3 3B run on an RTX 4090 (24 GB)?
Yes. SmolLM3 3B needs about 3 GB at Q4_K_M, inside a 24 GB card, at an estimated 190 tokens/sec.
How do I run SmolLM3 3B locally?
Install Ollama and run `ollama run smollm3:3b`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.