Nemotron 3.5 Super VL 120B-A12B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
The largest Nemotron 3.5: a vision-language MoE for reasoning, coding, tool use, document and video understanding. 120B total, 12B active, in the same hybrid stack of Mamba-2, MoE and a minority of attention layers, with a C-RADIO vision encoder; up to 1M tokens of context. Bolt 74B is compressed from it. NVIDIA publishes BF16 and NVFP4 checkpoints, and ggml-org a GGUF.
Nemotron 3.5 Super VL 120B-A12B needs about 73 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
VRAM and speed by quantization
Quoted against NVIDIA RTX 4090 (24 GB). Weights plus overhead only: this model's architecture is unpublished, so no KV cache is added.
| Quant | Memory | VRAM | Speed (est.) | Fit |
|---|---|---|---|---|
| Q2_K 2.63 bpw | 40.2 GB | ~16 tok/s | Offload | |
| Q3_K_M 3.41 bpw | 52 GB | ~14 tok/s | Offload | |
| Q4_K_M 4.83 bpw | 73.3 GB | — | Too big | |
| Q5_K_M 5.67 bpw | 85.9 GB | — | Too big | |
| Q6_K 6.56 bpw | 99.2 GB | — | Too big | |
| Q8_0 8.50 bpw | 128.3 GB | — | Too big | |
| F16 16.00 bpw | 240.8 GB | — | Too big |
Black marker = usable memory on the NVIDIA RTX 4090 (24 GB). Estimates from the memory-bandwidth roofline on the methodology page. Nemotron 3.5 Super VL 120B-A12B VRAM calculator →
Get Nemotron 3.5 Super VL 120B-A12B running
The cheapest catalogued GPU that runs Nemotron 3.5 Super VL 120B-A12B is the NVIDIA DGX Spark (128 GB).
or compare on Vast.ai from $0.77/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Nemotron 3.5 Super VL 120B-A12B
Install Ollama, then run:
ollama run nemotron-3.5-super:120bSpecifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source. Still unconfirmed: arch.kv_layers.
- Parameters
- 120 Billion (12B active)
- Context window
- 1M
- Architecture
- Hybrid Mamba-2 + MoE + attention, C-RADIO vision encoder
- Provider
- NVIDIA
- Licence
- OpenMDW-1.1
- Availability
- Weights downloadable
- Practical deployment
- Workstation (large unified memory or pro GPU)
- Specified at
- Q4_K_M
- System RAM
- 128 GB
- Record updated
- 2026-10-25
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- The KV-cache figure is an estimate: the attention-layer count was not read from a published config, so the cache is sized as if every layer attended. With only a minority of attention layers, the real cache is smaller.
See also Nemotron 3.5 Bolt 74B
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Can I run Nemotron 3.5 Super VL 120B-A12B on my GPU?
Other Nemotron 3.5 sizes
Nemotron 3.5 Super VL 120B-A12B — frequently asked questions
How much VRAM does Nemotron 3.5 Super VL 120B-A12B need?
About 73 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Nemotron 3.5 Super VL 120B-A12B run on an RTX 4090 (24 GB)?
No. Nemotron 3.5 Super VL 120B-A12B needs about 73 GB at Q4_K_M, more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.
How do I run Nemotron 3.5 Super VL 120B-A12B locally?
Install Ollama and run `ollama run nemotron-3.5-super:120b`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.
What other sizes does Nemotron 3.5 come in?
Nemotron 3.5 Lightning 30B-A3B (19 GB), Nemotron 3.5 Bolt 74B (23 GB), Nemotron 3.5 Super VL 120B-A12B (73 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.