Nemotron 3 Super 120B-A12B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
120B total / 12B active MoE. The low active-parameter count means throughput closer to a 12B model than a 120B one, but the full weight set still has to be resident — roughly 65 GB at Q4_K_M, so dual RTX 3090/4090 or a single 80GB datacenter card. Q8_0 and BF16 tags are published for workstations with more headroom. NVIDIA releases open training data and recipes with this family, which is unusual among open-weight releases.
Nemotron 3 Super 120B-A12B needs about 73 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 40.2 GB | ~16 tok/s | Offload | |
| Q3_K_M 3.41 bpw | 52 GB | ~14 tok/s | Offload | |
| Q4_K_M 4.83 bpw | 73.3 GB | — | Too big | |
| Q5_K_M 5.67 bpw | 85.9 GB | — | Too big | |
| Q6_K 6.56 bpw | 99.2 GB | — | Too big | |
| Q8_0 8.50 bpw | 128.3 GB | — | Too big | |
| F16 16.00 bpw | 240.8 GB | — | Too big |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Nemotron 3 Super 120B-A12B VRAM calculator →
Get Nemotron 3 Super 120B-A12B running
The cheapest catalogued GPU that runs Nemotron 3 Super 120B-A12B is the NVIDIA DGX Spark (128 GB).
or compare on Vast.ai from $0.77/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Nemotron 3 Super 120B-A12B
Install Ollama, then run:
ollama run nemotron-3-super:120b-a12bSpecifications
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
Preview — The model is released, but these specs are thin or rest on a single source. Individual fields may be wrong.
- Parameters
- 120 Billion (12B active)
- Context window
- 128,000
- Architecture
- Mixture-of-Experts (reasoning-trace)
- Provider
- NVIDIA
- Licence
- NVIDIA Open Model License
- Practical deployment
- Workstation (large unified memory or pro GPU)
- Specified at
- Q4_K_M
- System RAM
- 128 GB
- Record updated
- 2026-08-15
Weights are downloadable and commercial use is permitted, subject to the licence’s acceptable-use terms.
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Nemotron 3 Super 120B-A12B on my GPU?
- Nemotron 3 Super on AMD Ryzen AI Max+ 395
- Nemotron 3 Super on Apple M1 Max
- Nemotron 3 Super on Apple M1 Pro
- Nemotron 3 Super on Apple M1 Ultra
- Nemotron 3 Super on Apple M2 Max
- Nemotron 3 Super on Apple M2 Pro
- Nemotron 3 Super on Apple M3 Max
- Nemotron 3 Super on Apple M3 Pro
- Nemotron 3 Super on Apple M4
- Nemotron 3 Super on Apple M4 Max
- Nemotron 3 Super on Apple M5
- Nemotron 3 Super on Apple M5 Max
- Nemotron 3 Super on Apple M5 Pro
- Nemotron 3 Super on NVIDIA A100 80GB (PCIe)
- Nemotron 3 Super on NVIDIA DGX Spark
- Nemotron 3 Super on NVIDIA GeForce RTX 5090
- Nemotron 3 Super on NVIDIA H100 80GB (PCIe)
- Nemotron 3 Super on NVIDIA L40S
- Nemotron 3 Super on NVIDIA RTX 6000 Ada Generation
- Nemotron 3 Super on NVIDIA RTX PRO 6000 Blackwell
Nemotron 3 Super 120B-A12B — frequently asked questions
How much VRAM does Nemotron 3 Super 120B-A12B need?
About 73 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Nemotron 3 Super 120B-A12B run on an RTX 4090 (24 GB)?
No. Nemotron 3 Super 120B-A12B needs about 73 GB at Q4_K_M, more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.
How do I run Nemotron 3 Super 120B-A12B locally?
Install Ollama and run `ollama run nemotron-3-super:120b-a12b`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.