NVIDIA RTX 6000 Ada Generation for local LLMs
Written by Jakub Rusinowski · Last updated
With 48 GB at 960 GB/s, the RTX 6000 Ada Generation runs 118 catalogued models at Q4_K_M with 8K context. The largest that fits is Llama 3.3 70B Instruct (~45.7 GB), and the top pick is Cosmos 3 Super at 26–50 tok/s.
The workstation counterpart of the RTX 4090: same AD102 silicon with 48 GB ECC GDDR6, a 300W blower cooler that fits standard towers, and pro drivers. 48 GB runs Llama 3.3 70B at Q4 on a single card — the quietest path to 70B-class models in an office. Typical street price well below launch MSRP since the Blackwell generation arrived.
Models that run on the RTX 6000 Ada Generation
Q4_K_M, 8K context, 48 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 48 GB | VRAM | Speed |
|---|---|---|---|
| Cosmos 3 Super Cosmos 3 | 39.4 GB | 26–50 tok/s | |
| K2 Horizon MoVA 36B-A4B K2 Horizon | 25 GB | 101–194 tok/s | |
| Qwen 3.6 35B-A3B Qwen 3.6 | 22.1 GB | 153–294 tok/s | |
| Nex-N2.5 mini Nex-N2.5 | 21.9 GB | 107–206 tok/s | |
| Nex-N2 mini Nex-N2 | 21.9 GB | 107–206 tok/s | |
| Qwen 3.5 35B-A3B Qwen 3.5 | 21.9 GB | 107–206 tok/s | |
| Laguna XS 2.1 33B-A3B Poolside Laguna XS 2.1 | 20.7 GB | 108–207 tok/s | |
| Qwen 3 32B Qwen 3 | 22.8 GB | 25–49 tok/s | |
| Aya Expanse 32B Aya Expanse | 21.6 GB | 27–51 tok/s | |
| EXAONE 3.5 32B EXAONE 3.5 | 22.3 GB | 26–50 tok/s | |
| Cogito v1 32B Cogito v1 | 22.3 GB | 26–50 tok/s | |
| Granite 4.0 Small-H 32B-A9B IBM Granite 4.0 | 20.1 GB | 66–126 tok/s |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-07-21.
- Memory
- 48 GB
- Memory bandwidth
- 960 GB/s
- Architecture
- Ada Lovelace AD102
- Series
- Pro Workstation
- Board power
- 300 W
- Release year
- 2022
- Launch price
- $6,799
- Usable for models
- 48 GB
Similar GPUs
Frequently asked questions
Can the NVIDIA RTX 6000 Ada Generation run local LLMs?
Yes. With 48 GB (48 GB usable by a model) it runs 118 of the catalogued models at Q4_K_M with 8K context; the largest is Llama 3.3 70B Instruct, needing about 45.7 GB.
How fast is the NVIDIA RTX 6000 Ada Generation for AI inference?
It is estimated to run Llama 3.1 8B at 83–159 tok/s at Q4_K_M. Llama 3.3 70B is estimated at 13–26 tok/s. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.
What LLMs can I run on 48 GB?
Among the best that fit: Cosmos 3 Super, K2 Horizon MoVA 36B-A4B, Qwen 3.6 35B-A3B, Nex-N2.5 mini, Nex-N2 mini. The quickest start is Ollama: ollama run llama3.3:70b.
Keep going
Can I run it on the RTX 6000 Ada Generation?
More for this card
Tools