Thinking Machines594B (17B active)NVFP4 + MXFP8 (mixed) 下约 394 GB 显存
许可证: Apache-2.0本地运行: 官方本地运行硬件: 实际仅限数据中心成熟度: 早期版本证据: 中等

证据: 基准测试结果由厂商自行报告

Inkling 2 Small — 显存、速度与本地部署

作者: Jakub Rusinowski · 最后更新:

The second-generation Inkling: a 594B mixture-of-experts with 17B active per token (8 of 384 experts), 48 layers of which 40 use a 1,024-token sliding window and 8 are global, image input and a 1M-token context. It was trained natively in NVFP4, and the 393.5 GB NVFP4 checkpoint is the one Thinking Machines recommends; an MXFP8 build (622.8 GB) is about 1-2 points better on evals. Apache-2.0; official vLLM and SGLang recipes.

Inkling 2 Small 在 NVFP4 + MXFP8 (mixed) 下约需 394 GB 显存——量化权重加框架开销,不含 KV 缓存。在 Apple Silicon 上,这部分来自统一内存。

按量化级别的显存与速度

计算基准:NVIDIA RTX 4090 (24 GB)。包含 8K 上下文的 KV 缓存,因此数值高于上方的主数字。

量化显存速度(估算)适配
NVFP4 + MXFP8 (mixed)
5.30 bpw
394.8 GB—放不下

黑色标记 = NVIDIA RTX 4090 (24 GB) 上的可用显存。 估算来自内存带宽屋顶线模型,详见 方法说明页. Inkling 2 Small 显存计算器 →

运行 Inkling 2 Small

Inkling 2 Small is not practical on a single consumer GPU: it needs about 394.3 GB at Q4_K_M, more than any single device in our catalogue holds with headroom. How multi-GPU inference works, or rent a multi-GPU server by the hour:

立即云端部署 RunPod

或在 Vast.ai 比较

云 GPU 链接为推荐链接——我们可能获得佣金,您无需额外付费。

如何运行 Inkling 2 Small

Ollama lists Inkling 2 Small only as the cloud tag inkling-2-small:cloud, which runs on Ollama's servers, not your machine. Locally, Thinking Machines documents vLLM and SGLang on a multi-GPU server.

Hugging Face 上的权重: thinkingmachines/Inkling-2-Small ↗

规格

Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.

参数量
594 Billion (17B active)
上下文窗口
1,048,576
架构
Mixture-of-Experts (384 experts, 8 active), 40 sliding-window (1,024) + 8 global layers, vision encoder
提供商
Thinking Machines
许可证
Apache-2.0
Availability
Weights downloadable
Practical deployment
Multi-GPU server
规格量化
NVFP4 + MXFP8 (mixed)
系统内存
512 GB
记录更新于
2026-11-21
许可证Apache-2.0允许商业使用

Commercial use permitted. No usage restrictions beyond attribution.

限制与注意事项

  • Thinking Machines states at least 500 GB of aggregate GPU memory for the NVFP4 checkpoint (for example 2x B300 or 4x H200), and 800 GB for MXFP8.
  • The repository has no LICENSE file; Apache-2.0 is stated in the card's metadata and text.
  • Benchmarks on the card are vendor-reported.

另请参阅 Inkling Small

质量与使用场景

评分由模型作者或独立评测方发布——衡量质量而非吞吐量,并非我们实测。

最适合agentscodingvisionlong context

我的 GPU 能运行 Inkling 2 Small 吗?

Inkling 的其他尺寸

Inkling 2 Small — 常见问题

How much VRAM does Inkling 2 Small need?

About 394 GB at NVFP4 + MXFP8 (mixed) — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.

Does Inkling 2 Small run on an RTX 4090 (24 GB)?

No. Inkling 2 Small needs about 394 GB at NVFP4 + MXFP8 (mixed), more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.

How do I run Inkling 2 Small locally?

Ollama lists Inkling 2 Small only as the cloud tag inkling-2-small:cloud, which runs on Ollama's servers, not your machine. Locally, Thinking Machines documents vLLM and SGLang on a multi-GPU server. Running the published tag would send your prompts to a hosted GPU rather than your own machine.

What other sizes does Inkling come in?

Inkling (NVFP4) (589 GB), Inkling (BF16) (589 GB), Inkling Small (161 GB), Inkling 2 Small (394 GB), Inkling 2 Mini (21 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.