Written by Jakub Rusinowski · Last updated March 12, 2025
Model library → Gemma 3 → Gemma 3 4B Instruct
Outperforms Llama 3.1 8B on most benchmarks while requiring half the VRAM. The best small model for budget hardware. Runs on any GPU with 4GB+ VRAM.
Gemma 3 4B Instruct needs about 3 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
| Parameters | 4 Billion |
| Context window | 128,000 |
| Architecture | Dense |
| Provider | |
| Licence | Gemma Terms |
| Specified at | Q4_K_M |
| System RAM | 8 GB |
| Record updated | 2025-03-12 |
Gemma Terms — commercial use permitted. Weights are downloadable and commercial use is permitted, subject to the licence’s acceptable-use terms.
Modelled on a reference NVIDIA RTX 4090 (24 GB), at 8K context. Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.
| Quant | Weights | VRAM needed | Est. speed | Fit on 24 GB |
|---|---|---|---|---|
| Q2_K | 1.3 GB | 3.3 GB | ~211 tok/s (est.) | Fits comfortably |
| Q3_K_M | 1.7 GB | 3.6 GB | ~190 tok/s (est.) | Fits comfortably |
| Q4_K_M | 2.4 GB | 4.4 GB | ~161 tok/s (est.) | Fits comfortably |
| Q5_K_M | 2.8 GB | 4.8 GB | ~148 tok/s (est.) | Fits comfortably |
| Q6_K | 3.3 GB | 5.2 GB | ~136 tok/s (est.) | Fits comfortably |
| Q8_0 | 4.3 GB | 6.2 GB | ~116 tok/s (est.) | Fits comfortably |
| F16 | 8.0 GB | 9.9 GB | ~74 tok/s (est.) | Fits comfortably |
Want the memory numbers alone, at every quantization level and your own context length? Use the Gemma 3 4B Instruct VRAM calculator.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
The cheapest catalogued GPU that runs Gemma 3 4B Instruct is the Intel Arc B570 (10 GB).
Install Ollama, then run:
ollama run gemma3:4bWeights on Hugging Face: google/gemma-3-4b-it.
Best for: chat, edge devices, budget hardware, mobile.
← All Gemma 3 models | VRAM calculator | Check your own hardware