AMD Radeon AI PRO R9700 for local LLMs
Written by Jakub Rusinowski · Last updated
With 32 GB of GDDR6 at 640 GB/s, the AI PRO R9700 runs 122 catalogued models at Q4_K_M with 8K context. The largest that fits is K2 Horizon MoVA 36B-A4B (~25 GB), and the top pick is K2 Horizon MoVA 36B-A4B at 52–108 tok/s.
AMD's 32 GB workstation card: the same Navi 48 die as the RX 9070 XT with double the memory, 640 GB/s, 300 W and four DisplayPort 2.1 outputs. Two of them give 64 GB for well under the price of one 96 GB NVIDIA card. ROCm support is the thing to check before buying.
Models that run on the AI PRO R9700
Q4_K_M, 8K context, 32 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 32 GB | VRAM | Speed |
|---|---|---|---|
| K2 Horizon MoVA 36B-A4B K2 Horizon | 25 GB | 52–108 tok/s | |
| Qwen 3.6 35B-A3B Qwen 3.6 | 22.2 GB | 81–167 tok/s | |
| Nex-N2.5 mini Nex-N2.5 | 21.9 GB | 55–115 tok/s | |
| Nex-N2 mini Nex-N2 | 21.9 GB | 55–115 tok/s | |
| Qwen 3.5 35B-A3B Qwen 3.5 | 21.9 GB | 55–115 tok/s | |
| Laguna XS 2.1 33B-A3B Poolside Laguna XS 2.1 | 20.7 GB | 56–116 tok/s | |
| Qwen 3 32B Qwen 3 | 22.8 GB | 12–26 tok/s | |
| Aya Expanse 32B Aya Expanse | 21.6 GB | 13–27 tok/s | |
| EXAONE 3.5 32B EXAONE 3.5 | 22.3 GB | 13–26 tok/s | |
| Cogito v1 32B Cogito v1 | 22.3 GB | 13–26 tok/s | |
| Granite 4.0 Small-H 32B-A9B IBM Granite 4.0 | 20.1 GB | 33–68 tok/s | |
| Olmo 3 32B Olmo 3 | 20.1 GB | 13–27 tok/s |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-09-19.
- Memory
- 32 GB GDDR6
- Memory bandwidth
- 640 GB/s
- Memory bus
- 256-bit
- Architecture
- RDNA 4 Navi 48
- Series
- Radeon AI PRO
- Board power
- 300 W
- Release year
- 2025
- Launch price
- $1,299
- Compute backends
- ROCM, VULKAN
- Usable for models
- 32 GB
Similar GPUs
Frequently asked questions
Can the AMD Radeon AI PRO R9700 run local LLMs?
Yes. With 32 GB (32 GB usable by a model) it runs 122 of the catalogued models at Q4_K_M with 8K context; the largest is K2 Horizon MoVA 36B-A4B, needing about 25 GB.
How fast is the AMD Radeon AI PRO R9700 for AI inference?
It is estimated to run Llama 3.1 8B at 42–87 tok/s at Q4_K_M. Llama 3.3 70B does not fit: it needs about 44 GB against 32 GB usable. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.
What LLMs can I run on 32 GB?
Among the best that fit: K2 Horizon MoVA 36B-A4B, Qwen 3.6 35B-A3B, Nex-N2.5 mini, Nex-N2 mini, Qwen 3.5 35B-A3B.