证据: 基准测试结果由厂商自行报告
Inkling 2 Small — 显存、速度与本地部署
作者: Jakub Rusinowski · 最后更新:
The second-generation Inkling: a 594B mixture-of-experts with 17B active per token (8 of 384 experts), 48 layers of which 40 use a 1,024-token sliding window and 8 are global, image input and a 1M-token context. It was trained natively in NVFP4, and the 393.5 GB NVFP4 checkpoint is the one Thinking Machines recommends; an MXFP8 build (622.8 GB) is about 1-2 points better on evals. Apache-2.0; official vLLM and SGLang recipes.
Inkling 2 Small 在 NVFP4 + MXFP8 (mixed) 下约需 394 GB 显存——量化权重加框架开销,不含 KV 缓存。在 Apple Silicon 上,这部分来自统一内存。
按量化级别的显存与速度
计算基准:NVIDIA RTX 4090 (24 GB)。包含 8K 上下文的 KV 缓存,因此数值高于上方的主数字。
| 量化 | 显存 | 显存 | 速度(估算) | 适配 |
|---|---|---|---|---|
| NVFP4 + MXFP8 (mixed) 5.30 bpw | 394.8 GB | — | 放不下 |
黑色标记 = NVIDIA RTX 4090 (24 GB) 上的可用显存。 估算来自内存带宽屋顶线模型,详见 方法说明页. Inkling 2 Small 显存计算器 →
运行 Inkling 2 Small
Inkling 2 Small is not practical on a single consumer GPU: it needs about 394.3 GB at Q4_K_M, more than any single device in our catalogue holds with headroom. How multi-GPU inference works, or rent a multi-GPU server by the hour:
如何运行 Inkling 2 Small
Ollama lists Inkling 2 Small only as the cloud tag inkling-2-small:cloud, which runs on Ollama's servers, not your machine. Locally, Thinking Machines documents vLLM and SGLang on a multi-GPU server.
规格
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- 参数量
- 594 Billion (17B active)
- 上下文窗口
- 1,048,576
- 架构
- Mixture-of-Experts (384 experts, 8 active), 40 sliding-window (1,024) + 8 global layers, vision encoder
- 提供商
- Thinking Machines
- 许可证
- Apache-2.0
- Availability
- Weights downloadable
- Practical deployment
- Multi-GPU server
- 规格量化
- NVFP4 + MXFP8 (mixed)
- 系统内存
- 512 GB
- 记录更新于
- 2026-11-21
Commercial use permitted. No usage restrictions beyond attribution.
限制与注意事项
- Thinking Machines states at least 500 GB of aggregate GPU memory for the NVFP4 checkpoint (for example 2x B300 or 4x H200), and 800 GB for MXFP8.
- The repository has no LICENSE file; Apache-2.0 is stated in the card's metadata and text.
- Benchmarks on the card are vendor-reported.
另请参阅 Inkling Small
质量与使用场景
评分由模型作者或独立评测方发布——衡量质量而非吞吐量,并非我们实测。
我的 GPU 能运行 Inkling 2 Small 吗?
Inkling 的其他尺寸
Inkling 2 Small — 常见问题
How much VRAM does Inkling 2 Small need?
About 394 GB at NVFP4 + MXFP8 (mixed) — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Inkling 2 Small run on an RTX 4090 (24 GB)?
No. Inkling 2 Small needs about 394 GB at NVFP4 + MXFP8 (mixed), more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.
How do I run Inkling 2 Small locally?
Ollama lists Inkling 2 Small only as the cloud tag inkling-2-small:cloud, which runs on Ollama's servers, not your machine. Locally, Thinking Machines documents vLLM and SGLang on a multi-GPU server. Running the published tag would send your prompts to a hosted GPU rather than your own machine.
What other sizes does Inkling come in?
Inkling (NVFP4) (589 GB), Inkling (BF16) (589 GB), Inkling Small (161 GB), Inkling 2 Small (394 GB), Inkling 2 Mini (21 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.