NVIDIA Tesla V100 32GB (PCIe) for local LLMs
Written by Jakub Rusinowski · Last updated
With 32 GB of HBM2 at 900 GB/s, the Tesla V100 32GB (PCIe) runs 139 catalogued models at Q4_K_M with 8K context. The largest that fits is K2 Horizon MoVA 36B-A4B (~25 GB), and the top pick is K2 Horizon MoVA 36B-A4B.
32 GB of HBM2 at 900 GB/s and 250 W (PCIe). Passively cooled, so it needs server airflow or an aftermarket fan. llama.cpp runs on it; vLLM has caveats, and TensorRT-LLM and TGI have dropped Volta (sm_70). Used prices have ranged widely (roughly $650 to $1,300 in 2025–26), so no price is shown. No speed calibration for Volta: fit verdicts only, no tok/s.
Models that run on the Tesla V100 32GB (PCIe)
Q4_K_M, 8K context, 32 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 32 GB | VRAM | Speed |
|---|---|---|---|
| K2 Horizon MoVA 36B-A4B K2 Horizon | 25 GB | — | |
| Qwen 3.6 35B-A3B Qwen 3.6 | 22.2 GB | — | |
| Nex-N2.5 mini Nex-N2.5 | 21.9 GB | — | |
| Nex-N2 mini Nex-N2 | 21.9 GB | — | |
| Qwen 3.5 35B-A3B Qwen 3.5 | 21.9 GB | — | |
| Jais 3 32B Jais 3 | 24 GB | — | |
| Laguna XS 2.1 33B-A3B Poolside Laguna XS 2.1 | 20.7 GB | — | |
| EXAONE 4.5 33B EXAONE 4 | 22.3 GB | — | |
| Qwen 3 32B Qwen 3 | 22.8 GB | — | |
| Aya Expanse 32B Aya Expanse | 21.6 GB | — | |
| Olmo 3.1 32B Instruct Olmo 3 | 21.7 GB | — | |
| Olmo 3.1 32B Think Olmo 3 | 21.7 GB | — |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-11-11.
- Memory
- 32 GB HBM2
- Memory bandwidth
- 900 GB/s
- Memory bus
- 4096-bit
- Architecture
- Volta GV100
- Series
- Tesla
- Board power
- 250 W
- Release year
- 2018
- Compute backends
- CUDA, VULKAN
- Usable for models
- 32 GB
Similar GPUs
Frequently asked questions
Can the NVIDIA Tesla V100 32GB (PCIe) run local LLMs?
Yes. With 32 GB (32 GB usable by a model) it runs 139 of the catalogued models at Q4_K_M with 8K context; the largest is K2 Horizon MoVA 36B-A4B, needing about 25 GB.
How fast is the NVIDIA Tesla V100 32GB (PCIe) for AI inference?
This site has no speed calibration for the Volta GV100 architecture, so it publishes fit verdicts for this card but no tokens-per-second estimate. Llama 3.3 70B does not fit: it needs about 44 GB against 32 GB usable.
What LLMs can I run on 32 GB?
Among the best that fit: K2 Horizon MoVA 36B-A4B, Qwen 3.6 35B-A3B, Nex-N2.5 mini, Nex-N2 mini, Qwen 3.5 35B-A3B.