IBM Granite Switch 4.1 — IBM 的本地 AI 模型
作者： Jakub Rusinowski · 最后更新：
组合式 Granite 4.1 检查点：基础模型、用于决策和 guardian 检查的 aLoRA 适配器以及语音编码器打包在一个文件中。控制 token 按请求切换模式，因此一个端点即可聊天、决策和筛查。Apache-2.0，提供官方 GGUF。
变体
IBM Granite Switch 4.1 最小的变体在 Q4_K_M 下约需 6 GB 显存——量化权重加框架开销，不含 KV 缓存。
| 模型 | Q4 下显存 | 显存 | 上下文 | 运行 |
|---|---|---|---|---|
| Granite Switch 4.1 8B → 8.8B | ~6.1 GB | 131,072 | 云端托管标签 | |
| Granite Switch 4.1 30B → 30B | ~18.9 GB | 131,072 | 云端托管标签 |
显存为 Q4_K_M 下的量化权重加开销，与 GPU 与显存检测器使用同一引擎计算。
如何在本地运行 IBM Granite Switch 4.1
安装 Ollama，然后拉取标签。
No Ollama tag. Use IBM's official GGUF with llama.cpp.
在上方选择一个尺寸，查看它自己的显存、速度估算和安装命令。
许可证
Commercial use permitted. No usage restrictions beyond attribution.
适用于: Granite Switch 4.1 8B, Granite Switch 4.1 30B推荐 GPU
目录中能在本地运行 IBM Granite Switch 4.1（至少 6 GB 显存）的最便宜 GPU 是 Intel Arc B570 (10 GB).
IBM Granite Switch 4.1 — 常见问题
How much VRAM does IBM Granite Switch 4.1 need?
IBM Granite Switch 4.1 needs about 6 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: Granite Switch 4.1 8B (6 GB, Q4_K_M); Granite Switch 4.1 30B (19 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run IBM Granite Switch 4.1 on an RTX 4090 (24 GB)?
Yes — IBM Granite Switch 4.1 runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
What quantization should I use for IBM Granite Switch 4.1?
Q4_K_M is the best balance of quality and VRAM for IBM Granite Switch 4.1 in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
How do I run IBM Granite Switch 4.1 with Ollama?
IBM Granite Switch 4.1 has no local Ollama tag — the published tag is cloud-hosted, so running it sends your prompts to a hosted GPU rather than your own machine.