DeepSeek-OCR 3B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
All 3B parameters stay resident — about 2.6 GB at Q4_K_M, so it runs on effectively any modern GPU — while the MoE decoder activates 6 of 64 routed experts plus 2 shared, roughly 570M parameters per token. Token budgets are explicit and chosen per page: Tiny 64 tokens at 512x512, Small 100 at 640x640, Base 256 at 1024x1024, Large 400 at 1280x1280, and a Gundam mode composing several 640x640 views with one 1024x1024 global view. 8,192-token sequence length. MIT licensed.
DeepSeek-OCR 3B needs about 3 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 1.8 GB | ~328 tok/s | Fits | |
| Q3_K_M 3.41 bpw | 2.1 GB | ~320 tok/s | Fits | |
| Q4_K_M 4.83 bpw | 2.6 GB | ~307 tok/s | Fits | |
| Q5_K_M 5.67 bpw | 2.9 GB | ~300 tok/s | Fits | |
| Q6_K 6.56 bpw | 3.3 GB | ~292 tok/s | Fits | |
| Q8_0 8.50 bpw | 4 GB | ~278 tok/s | Fits | |
| F16 16.00 bpw | 6.8 GB | ~232 tok/s | Fits |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. DeepSeek-OCR 3B VRAM calculator →
Get DeepSeek-OCR 3B running
The cheapest catalogued GPU that runs DeepSeek-OCR 3B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run DeepSeek-OCR 3B
Install Ollama, then run:
ollama run deepseek-ocrSpecifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 3 Billion (570M active)
- Context window
- 8,192
- Architecture
- DeepEncoder + DeepSeek-3B MoE decoder
- Provider
- DeepSeek
- Licence
- MIT
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- Q4_K_M
- System RAM
- 8 GB
- Record updated
- 2026-09-11
Commercial use permitted. No usage restrictions beyond attribution.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run DeepSeek-OCR 3B on my GPU?
DeepSeek-OCR 3B — frequently asked questions
How much VRAM does DeepSeek-OCR 3B need?
About 3 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does DeepSeek-OCR 3B run on an RTX 4090 (24 GB)?
Yes. DeepSeek-OCR 3B needs about 3 GB at Q4_K_M, inside a 24 GB card, at an estimated 307 tokens/sec.
How do I run DeepSeek-OCR 3B locally?
Install Ollama and run `ollama run deepseek-ocr`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.