NVIDIADesktop GPUCurrent

NVIDIA RTX PRO 4500 Blackwell (Workstation Edition) for local LLMs

Written by Jakub Rusinowski · Last updated

With 32 GB of GDDR7 at 896 GB/s, the RTX PRO 4500 Blackwell (Workstation Edition) runs 139 catalogued models at Q4_K_M with 8K context. The largest that fits is K2 Horizon MoVA 36B-A4B (~25 GB), and the top pick is K2 Horizon MoVA 36B-A4B at 86–179 tok/s.

32 GB of GDDR7 with ECC on a 256-bit bus: 896 GB/s at 200 W, dual slot, 10,496 CUDA cores. This is the Workstation Edition. A passively cooled Server Edition is quoted at 800 GB/s and 165 W; it is a different record and is not listed until a second source confirms it.

Models that run on the RTX PRO 4500 Blackwell (Workstation Edition)

Q4_K_M, 8K context, 32 GB usable. Ranked by quality and speed.

ModelVRAMSpeed
K2 Horizon MoVA 36B-A4B
K2 Horizon
25 GB86–179 tok/s
Qwen 3.6 35B-A3B
Qwen 3.6
22.2 GB132–274 tok/s
Nex-N2.5 mini
Nex-N2.5
21.9 GB92–190 tok/s
Nex-N2 mini
Nex-N2
21.9 GB92–190 tok/s
Qwen 3.5 35B-A3B
Qwen 3.5
21.9 GB92–190 tok/s
Jais 3 32B
Jais 3
24 GB20–41 tok/s
Laguna XS 2.1 33B-A3B
Poolside Laguna XS 2.1
20.7 GB92–192 tok/s
EXAONE 4.5 33B
EXAONE 4
22.3 GB21–45 tok/s
Qwen 3 32B
Qwen 3
22.8 GB21–43 tok/s
Aya Expanse 32B
Aya Expanse
21.6 GB22–46 tok/s
Olmo 3.1 32B Instruct
Olmo 3
21.7 GB22–45 tok/s
Olmo 3.1 32B Think
Olmo 3
21.7 GB22–45 tok/s
Showing 12 of 139

Buy it or rent the same memory

Buy the card, or rent a GPU with the same memory by the hour to try models first.

Buy this hardware NVIDIA GeForce RTX 5090 32GB — 32 GB VRAM · 575 W board powerDeploy in the cloud now NVIDIA A40 on RunPod — from $0.44/hr · rate checked 2026-08

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Speed vs other GPUs

Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated

Specifications

Specs last updated 2026-10-07.

Memory
32 GB GDDR7
Memory bandwidth
896 GB/s
Memory bus
256-bit
Architecture
Blackwell GB203
Series
RTX PRO Blackwell
Board power
200 W
Release year
2025
Compute backends
CUDA, VULKAN
Usable for models
32 GB
Best for32 GB workstation AI30B-class modelsECC memory

Similar GPUs

Frequently asked questions

Can the NVIDIA RTX PRO 4500 Blackwell (Workstation Edition) run local LLMs?

Yes. With 32 GB (32 GB usable by a model) it runs 139 of the catalogued models at Q4_K_M with 8K context; the largest is K2 Horizon MoVA 36B-A4B, needing about 25 GB.

How fast is the NVIDIA RTX PRO 4500 Blackwell (Workstation Edition) for AI inference?

It is estimated to run Llama 3.1 8B at 70–145 tok/s at Q4_K_M. Llama 3.3 70B does not fit: it needs about 44 GB against 32 GB usable. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.

What LLMs can I run on 32 GB?

Among the best that fit: K2 Horizon MoVA 36B-A4B, Qwen 3.6 35B-A3B, Nex-N2.5 mini, Nex-N2 mini, Qwen 3.5 35B-A3B.