Inkling (BF16) — VRAM, Speed & Local Setup

Written by Jakub Rusinowski · Last updated July 16, 2026

Model library → Inkling → Inkling (BF16)

The full-precision reference checkpoint. BF16 weights need roughly 2 TB of VRAM — a Hopper-or-later multi-GPU cluster — so this is the build for research, evaluation and fine-tuning where maximum fidelity matters, not for cost-sensitive serving (use the NVFP4 build for that). Same 975B / 41B-active multimodal MoE, 1M-token context, native text / image / audio input, and MTP drafter layers. Runs under Transformers 5.14+, SGLang and vLLM. Apache 2.0. Specs from launch coverage — verify on the Hugging Face model card.

Inkling (BF16) needs about 589 GB of VRAM at BF16 (full precision) — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.

Specifications

Parameters	975 Billion (41B active)
Context window	1,000,000
Architecture	Multimodal Mixture-of-Experts — 256 experts, top-6 routed + 2 shared; BF16
Provider	Thinking Machines
Licence	Apache 2.0
Specified at	BF16 (full precision)
System RAM	2048 GB
Record updated	2026-07-16

Licence

Apache-2.0 — commercial use permitted. Commercial use permitted. No usage restrictions beyond attribution.

VRAM and Speed by Quantization

Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.

Quant	Weights	VRAM needed	Est. speed	Fit on 24 GB
Q2_K	320.5 GB	321.3 GB	—	Won't fit
Q3_K_M	415.6 GB	416.4 GB	—	Won't fit
Q4_K_M	588.7 GB	589.5 GB	—	Won't fit
Q5_K_M	691.0 GB	691.8 GB	—	Won't fit
Q6_K	799.5 GB	800.3 GB	—	Won't fit
Q8_0	1035.9 GB	1036.7 GB	—	Won't fit
F16	1950.0 GB	1950.8 GB	—	Won't fit

Want the memory numbers alone, at every quantization level and your own context length? Use the Inkling (BF16) VRAM calculator.

Deploy in the Cloud NowRunPod

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

How to Run Inkling (BF16)

Install Ollama, then run:

ollama run inkling

Weights on Hugging Face: thinkingmachines/Inkling.

Published Benchmark Scores

Quality scores as published by the model's authors or an independent evaluator — not throughput, and not measured by us.

Benchmark	Score	Provenance
AIME 2026	97.1 %	vendor-claimed · Thinking Machines (launch)
GPQA Diamond	87.2 %	vendor-claimed · Thinking Machines (launch)
SWE-bench Verified	77.6 %	vendor-claimed · Thinking Machines (launch)
MMMU Pro (Standard 10)	73.3 %	vendor-claimed · Thinking Machines (launch)
VoiceBench	91.4 %	vendor-claimed · Thinking Machines (launch)

Best for: multimodal, research, fine tuning, long context, enterprise.

Can I Run Inkling (BF16) on My GPU?

Inkling on Apple M3 Ultra

Other Inkling Sizes

Inkling (NVFP4) — about 589 GB VRAM at Q4_K_M

Inkling (BF16) — Frequently Asked Questions

How much VRAM does Inkling (BF16) need?

About 589 GB at BF16 (full precision) — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.

Does Inkling (BF16) run on an RTX 4090 (24 GB)?

No. Inkling (BF16) needs about 589 GB at BF16 (full precision), more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.

How do I run Inkling (BF16) locally?

Install Ollama and run `ollama run inkling`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.

What other sizes does Inkling come in?

Inkling (NVFP4) (589 GB), Inkling (BF16) (589 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.

← All Inkling models | VRAM calculator | Check your own hardware