VRAM Calculator for Local LLMs

Pick any model to see how much GPU VRAM it needs at each quantization level — from aggressive Q2 up to full FP16. Estimates are weights + KV cache + framework overhead from the same memory-bandwidth model that powers the rest of the site, so they line up with our GPU checker and model pages.

How the estimate works

VRAM ≈ quantized weight size (parameters × bits-per-weight ÷ 8) + KV cache for your context length + ~0.8 GB of CUDA/framework overhead. Q4_K_M is the usual quality/VRAM sweet spot.

Popular models

• BitNet b1.58 3B VRAM requirements — Microsoft
• DeepSeek R1 Distill Llama 8B VRAM requirements — DeepSeek
• DeepSeek R1 Distill Qwen 32B VRAM requirements — DeepSeek
• DeepSeek R1 Distill Qwen 14B VRAM requirements — DeepSeek
• DeepSeek R1 (671B) VRAM requirements — DeepSeek
• Llama 3.3 70B Instruct VRAM requirements — Meta
• Llama 3.1 8B Instruct VRAM requirements — Meta
• Nemotron 70B Instruct VRAM requirements — NVIDIA
• Command R (35B) VRAM requirements — Cohere
• Command R+ (104B) VRAM requirements — Cohere
• Phi-4 (14B) VRAM requirements — Microsoft
• Phi 3.5 Mini VRAM requirements — Microsoft
• Qwen 2.5 Coder 32B VRAM requirements — Alibaba Cloud
• Qwen 2.5 14B Instruct VRAM requirements — Alibaba Cloud
• Qwen 2.5 7B Instruct VRAM requirements — Alibaba Cloud
• Qwen 2.5 72B Instruct VRAM requirements — Alibaba Cloud
• Gemma 2 9B IT VRAM requirements — Google
• Mistral Small 3 (24B) VRAM requirements — Mistral AI
• Mistral NeMo 12B VRAM requirements — Mistral AI
• Yi 1.5 34B Chat VRAM requirements — 01.AI
• Yi 1.5 9B Chat VRAM requirements — 01.AI
• Granite 3.0 8B Instruct VRAM requirements — IBM
• Llama 4 Scout 17B VRAM requirements — Meta
• Llama 4 Maverick 17B VRAM requirements — Meta