Evidence: benchmark results vendor-reported
Granite Switch 4.1 30B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
The same composed layout on the Granite 4.1 30B base: 30.0B parameters in one checkpoint, 66 layers, 128K context. Official GGUF.
Granite Switch 4.1 30B needs about 19 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Includes the KV cache at 8K context, so it reads higher than the headline figure.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 12.9 GB | ~60 tok/s | Fits | |
| Q3_K_M 3.41 bpw | 15.8 GB | ~48 tok/s | Fits | |
| Q4_K_M 4.83 bpw | 21.1 GB | ~36 tok/s | Fits | |
| Q5_K_M 5.67 bpw | 24.3 GB | ~5 tok/s | Offload | |
| Q6_K 6.56 bpw | 27.6 GB | ~4 tok/s | Offload | |
| Q8_0 8.50 bpw | 34.9 GB | ~3 tok/s | Offload | |
| F16 16.00 bpw | 63 GB | — | Too big |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Granite Switch 4.1 30B VRAM calculator →
Get Granite Switch 4.1 30B running
The cheapest catalogued GPU that runs Granite Switch 4.1 30B is the AMD Radeon RX 7900 XTX (24 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Granite Switch 4.1 30B
No Ollama tag. Use IBM's official GGUF with llama.cpp.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 30.0 Billion (base + adapters + encoder)
- Context window
- 131,072
- Architecture
- Dense Transformer (GQA) with aLoRA adapters
- Provider
- IBM
- Licence
- Apache-2.0
- Practical deployment
- Single consumer GPU
- Specified at
- Q4_K_M
- System RAM
- 32 GB
- Record updated
- 2026-11-03
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- A composed checkpoint: the adapters are reached through control tokens.
- Kept out of general chat rankings; compare it with Granite 4.1 30B for plain chat.
See also Granite 4.1 30B
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Other IBM Granite Switch 4.1 sizes
Granite Switch 4.1 30B — frequently asked questions
How much VRAM does Granite Switch 4.1 30B need?
About 19 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Granite Switch 4.1 30B run on an RTX 4090 (24 GB)?
Yes. Granite Switch 4.1 30B needs about 19 GB at Q4_K_M, inside a 24 GB card, at an estimated 36 tokens/sec.
How do I run Granite Switch 4.1 30B locally?
No Ollama tag. Use IBM's official GGUF with llama.cpp. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does IBM Granite Switch 4.1 come in?
Granite Switch 4.1 8B (6 GB), Granite Switch 4.1 30B (19 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.