Nemotron Labs Dory 1.0 4B — VRAM, speed & local setup
Written by Jakub Rusinowski · Last updated
A 4B looped model: an input block, a recurrent block applied one to four times per token, and an output block, in a normalized-GPT stack that interleaves sliding-window and full attention. NVIDIA recommends two loops; four help most on agentic tasks such as Terminal-Bench and SWE-bench. Up to 256K tokens of context, reasoning and tool use, OpenMDW-1.1. The local build is NVIDIA's Q5_K_M GGUF (3.67 GB), and it needs a Dory-enabled llama.cpp fork until ggml-org/llama.cpp#30306 lands.
Nemotron Labs Dory 1.0 4B needs about 4 GB of VRAM at Q5_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.
Get Nemotron Labs Dory 1.0 4B running
The cheapest catalogued GPU that runs Nemotron Labs Dory 1.0 4B is the Intel Arc B570 (10 GB).
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
How to run Nemotron Labs Dory 1.0 4B
No Ollama build. Dory needs a llama.cpp fork with looped-layer support (ggml-org/llama.cpp#30306), which Ollama does not ship.
Specifications
Verified — Checked against the primary source — the model card or the vendor spec page — and corroborated by a second independent source.
- Parameters
- 4 Billion
- Context window
- 262,144
- Architecture
- Looped (latent-recurrent) normalized-GPT, interleaved SWA and full attention
- Provider
- NVIDIA
- Licence
- OpenMDW-1.1
- Availability
- Weights downloadable
- Practical deployment
- Edge device (phone, SBC)
- Specified at
- Q5_K_M
- System RAM
- 16 GB
- Record updated
- 2026-10-25
- Runs with
- llama.cpp: pending source
Commercial use permitted. No usage restrictions beyond attribution.
Limits and caveats
- No speed estimate and no fit verdict: the recurrent block runs up to four times per token, so the bytes read per token are not the weights once, which is what our estimator assumes.
- The memory figure is weights plus overhead at zero context. With per-loop KV caching the cache is stored once per loop; NVIDIA's last-loop mode keeps one.
- Mainline llama.cpp does not run it yet: the card builds a Dory-enabled fork (ggml-org/llama.cpp#30306).
Quality and use cases
Scores as published by the model’s authors or an independent evaluator — quality, not throughput, and not measured by us.
Nemotron Labs Dory 1.0 4B — frequently asked questions
How much VRAM does Nemotron Labs Dory 1.0 4B need?
About 4 GB at Q5_K_M — quantized weights plus framework overhead, before any KV cache. We publish no context-length figure or speed for it: its looped layers re-run a shared block up to four times per token, so the bytes read per token are not the weights once, which is what our speed estimate assumes. Apple Silicon counts unified memory toward the same figure.
Does Nemotron Labs Dory 1.0 4B run on an RTX 4090 (24 GB)?
Its weights and overhead come to about 4 GB at Q5_K_M, but we do not publish a fit verdict or a speed for it: its looped layers re-run a shared block up to four times per token, so the bytes read per token are not the weights once, which is what our speed estimate assumes.
How do I run Nemotron Labs Dory 1.0 4B locally?
No Ollama build. Dory needs a llama.cpp fork with looped-layer support (ggml-org/llama.cpp#30306), which Ollama does not ship. Running the published tag would send your prompts to a hosted GPU rather than your own machine.