Nemotron Labs Dory — local AI model by NVIDIA
Written by Jakub Rusinowski · Last updated
NVIDIA's research line of looped (latent-recurrent) language models. Dory re-runs a shared block of layers one to four times per token, spending more compute on each token instead of adding parameters or generating longer answers. We publish its memory but no speed: looping changes how many bytes are read per token, which our estimator does not model.
Variants
The smallest Nemotron Labs Dory variant needs about 4 GB of VRAM at Q5_K_M — quantized weights plus framework overhead, before any KV cache.
| Model | VRAM at Q4 | VRAM | Context | Run it |
|---|---|---|---|---|
| Nemotron Labs Dory 1.0 4B → 4B | ~4.5 GB | 262,144 | cloud-hosted tag |
Memory is quantized weights plus overhead at Q4_K_M, from the same engine as the GPU & VRAM checker.
How to run Nemotron Labs Dory locally
Install Ollama, then pull the tag.
No Ollama build. Dory needs a llama.cpp fork with looped-layer support (ggml-org/llama.cpp#30306), which Ollama does not ship.
Pick a size above for its own VRAM figure, speed estimate and install command.
Licence
Commercial use permitted. No usage restrictions beyond attribution.
Applies to: Nemotron Labs Dory 1.0 4BRecommended GPU
The cheapest catalogued GPU that runs Nemotron Labs Dory locally (min 4 GB VRAM) is the Intel Arc B570 (10 GB).
Nemotron Labs Dory — frequently asked questions
How much VRAM does Nemotron Labs Dory need?
Nemotron Labs Dory needs about 4 GB VRAM at Q5_K_M quantization for its smallest variant. Variants: Nemotron Labs Dory 1.0 4B (4 GB, Q5_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run Nemotron Labs Dory on an RTX 4090 (24 GB)?
Yes — Nemotron Labs Dory runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q5_K_M.
What quantization should I use for Nemotron Labs Dory?
Nemotron Labs Dory is published only pre-quantised, so there is no Q4_K_M or Q8_0 build to choose between: each variant ships in the format listed beside it above (Q5_K_M). Use the vendor's runtime for that format, as the variant pages describe.
How do I run Nemotron Labs Dory with Ollama?
Nemotron Labs Dory has no local Ollama tag — the published tag is cloud-hosted, so running it sends your prompts to a hosted GPU rather than your own machine.