IBM Granite Switch 4.1 — local AI model by IBM
Written by Jakub Rusinowski · Last updated
Composed Granite 4.1 checkpoints: a base model, aLoRA adapters for decisions and guardian checks, and a speech encoder in one file. Control tokens switch the mode per request, so one endpoint can chat, decide and screen. Apache-2.0, official GGUFs.
Variants
The smallest IBM Granite Switch 4.1 variant needs about 6 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache.
| Model | VRAM at Q4 | VRAM | Context | Run it |
|---|---|---|---|---|
| Granite Switch 4.1 8B → 8.8B | ~6.1 GB | 131,072 | cloud-hosted tag | |
| Granite Switch 4.1 30B → 30B | ~18.9 GB | 131,072 | cloud-hosted tag |
Memory is quantized weights plus overhead at Q4_K_M, from the same engine as the GPU & VRAM checker.
How to run IBM Granite Switch 4.1 locally
Install Ollama, then pull the tag.
No Ollama tag. Use IBM's official GGUF with llama.cpp.
Pick a size above for its own VRAM figure, speed estimate and install command.
Licence
Commercial use permitted. No usage restrictions beyond attribution.
Applies to: Granite Switch 4.1 8B, Granite Switch 4.1 30BRecommended GPU
The cheapest catalogued GPU that runs IBM Granite Switch 4.1 locally (min 6 GB VRAM) is the Intel Arc B570 (10 GB).
IBM Granite Switch 4.1 — frequently asked questions
How much VRAM does IBM Granite Switch 4.1 need?
IBM Granite Switch 4.1 needs about 6 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: Granite Switch 4.1 8B (6 GB, Q4_K_M); Granite Switch 4.1 30B (19 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run IBM Granite Switch 4.1 on an RTX 4090 (24 GB)?
Yes — IBM Granite Switch 4.1 runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
What quantization should I use for IBM Granite Switch 4.1?
Q4_K_M is the best balance of quality and VRAM for IBM Granite Switch 4.1 in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
How do I run IBM Granite Switch 4.1 with Ollama?
IBM Granite Switch 4.1 has no local Ollama tag — the published tag is cloud-hosted, so running it sends your prompts to a hosted GPU rather than your own machine.