Best Local LLMs for the NVIDIA RTX PRO 5000 Blackwell (72 GB)
Written by Jakub Rusinowski · Last updated October 7, 2026
Best all-round pick: GPT-OSS 120B
The NVIDIA RTX PRO 5000 Blackwell (72 GB) has 72 GB of VRAM, of which about 72 GB is available to a model. The largest model it can hold is GPT-OSS 120B (117B, 71.3 GB at Q4_K_M).
Best models overall on the NVIDIA RTX PRO 5000 Blackwell (72 GB)
| Model | Score | Memory at Q4_K_M | Context | Licence |
|---|---|---|---|---|
| GPT-OSS 120B | 96.4 | 71.3 GB | 128K | Apache-2.0 |
| GLM-5.1 72B | 94.7 | 44.3 GB | 125K | MIT |
| Qwen 3.5 72B | 94.4 | 44.3 GB | 125K | Apache 2.0 |
| Llama 3.3 70B Instruct | 93.8 | 43.1 GB | 128K | Llama Community |
| Cogito v1 70B | 92.6 | 43.1 GB | 128K | Apache-2.0 |
| Command R+ (104B) | 92.6 | 63.6 GB | 125K | CC-BY-NC |
Best model by what you are doing
| Workload | Recommended on this GPU |
|---|---|
| Coding | Qwen3-Coder-Next (80B-A3B MoE) (99) Qwen 3.5 72B (96.6) Llama 3.3 70B Instruct (96.1) |
| General assistant | GPT-OSS 120B (96.4) GLM-5.1 72B (94.7) Qwen 3.5 72B (94.4) |
| Reasoning | GLM-5.1 72B (100) Gemma 4 27B ⭐ (97.1) Qwen 3.5 72B (96.8) |
| RAG | Llama 4 Scout 17B (94.4) Gemma 4 31B (94.3) Qwen 3.6 27B (93.3) |
| Agents | Qwen3-Coder-Next (80B-A3B MoE) (95.2) Kolibri 1 (94.5) Ternary Bonsai 27B (90.4) |
| Vision | Qwen 3.5 72B (99.7) Llama 3.2 90B Vision Instruct (97.1) Qwen 3.7 35B-A3B (94.8) |
How these numbers are calculated
- Usable memory is 72 GB of the card's 72 GB.
- Fit is measured at Q4_K_M — quantized weights plus KV cache plus 0.8 GB runtime overhead.
- Within this card's budget, models are ranked on how well they use the memory available, not on being small — so the recommendation scales with the hardware.
FAQ
What is the best LLM for the NVIDIA RTX PRO 5000 Blackwell (72 GB)?
GPT-OSS 120B is the strongest all-round pick that fits its 72 GB.
What is the largest model the NVIDIA RTX PRO 5000 Blackwell (72 GB) can run?
GPT-OSS 120B — 117B parameters, needing 71.3 GB at Q4_K_M.
How much of the NVIDIA RTX PRO 5000 Blackwell (72 GB)'s memory can a model actually use?
About 72 GB of its 72 GB, before the desktop and runtime overhead are accounted for.
Similar GPUs
- Best models for the NVIDIA A100 80GB (PCIe)
- Best models for the NVIDIA H100 80GB (PCIe)
- Best models for the NVIDIA H100 80GB (SXM)
By Workload
- Best local LLMs for coding
- Best local LLMs for general assistant
- Best local LLMs for reasoning
- Best local LLMs for rag