Nemotron 3 Nano Omni 30B-A3B — VRAM, Speed & Local Setup

作者： Jakub Rusinowski · 最后更新： 2026年9月6日

Model library → Nemotron 3 Nano Omni → Nemotron 3 Nano Omni 30B-A3B

30B total, 3B active per token. Accepts text, image, speech and video and returns text. All 30B must be resident — 18.9 GB at Q4_K_M, so a 24 GB card carries it — while the per-token bandwidth cost is that of a 3B model, which is why it feels far quicker than its size suggests. 260K context, NVIDIA Open Model License.

Nemotron 3 Nano Omni 30B-A3B needs about 19 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.

Specifications

Parameters30 Billion (3B active)
Context window260,000
ArchitectureMixture-of-Experts (omni-modal: text, image, speech, video)
ProviderNVIDIA
LicenceNVIDIA Open Model License
Specified atQ4_K_M
System RAM32 GB
Record updated2026-09-06

Licence

NVIDIA Open Model License — commercial use permitted. Weights are downloadable and commercial use is permitted, subject to the licence’s acceptable-use terms.

VRAM and Speed by Quantization

Modelled on a reference NVIDIA RTX 4090 (24 GB), with no KV cache (this record has no published architecture). Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.

QuantWeightsVRAM neededEst. speedFit on 24 GB
Q2_K9.9 GB10.7 GB~212 tok/s (est.)Fits comfortably
Q3_K_M12.8 GB13.6 GB~196 tok/s (est.)Fits comfortably
Q4_K_M18.1 GB18.9 GB~172 tok/s (est.)Fits comfortably
Q5_K_M21.3 GB22.1 GB~160 tok/s (est.)Tight fit
Q6_K24.6 GB25.4 GB~22 tok/s (est.)Offloads to system RAM (slow)
Q8_031.9 GB32.7 GB~20 tok/s (est.)Offloads to system RAM (slow)
F1660.0 GB60.8 GB—Won't fit

Want the memory numbers alone, at every quantization level and your own context length? Use the Nemotron 3 Nano Omni 30B-A3B VRAM calculator.

Buy This HardwareAMD Radeon RX 7900 XT 20GB — 20 GB VRAM · 315 W board powerDeploy in the Cloud NowRTX 4090 on RunPod — from $0.34/hr · rate checked 2026-07

or compare on Vast.ai from $0.35/hr (typical low · varies)

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Recommended GPU

The cheapest catalogued GPU that runs Nemotron 3 Nano Omni 30B-A3B is the AMD Radeon RX 7900 XT (20 GB).

联盟营销声明: 本页部分链接为联盟推广链接——如果你通过它们购买，LLM Configurator 可能会获得佣金，而你无需支付任何额外费用。作为亚马逊联盟成员（Amazon Associate），LLM Configurator 会从符合条件的购买中获得收益。
AMD Radeon RX 7900 XT 20GB
20 GB VRAM · 315 W board power
2026年价格波动较大——请以当前商品页价格为准。
在亚马逊查看价格

How to Run Nemotron 3 Nano Omni 30B-A3B

Install Ollama, then run:

ollama run nemotron-3-nano-omni

Weights on Hugging Face: nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16.

Best for: multimodal, video, audio, agentic tasks, reasoning.

Can I Run Nemotron 3 Nano Omni 30B-A3B on My GPU?

Nemotron 3 Nano Omni 30B-A3B — Frequently Asked Questions

How much VRAM does Nemotron 3 Nano Omni 30B-A3B need?
About 19 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Nemotron 3 Nano Omni 30B-A3B run on an RTX 4090 (24 GB)?
Yes. Nemotron 3 Nano Omni 30B-A3B needs about 19 GB at Q4_K_M, inside a 24 GB card, at an estimated 172 tokens/sec.
How do I run Nemotron 3 Nano Omni 30B-A3B locally?
Install Ollama and run `ollama run nemotron-3-nano-omni`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.

← All Nemotron 3 Nano Omni models | VRAM calculator | Check your own hardware