NVIDIA H100 80GB — Local LLM Performance & Compatibility

Name: LLM Configurator — GPU VRAM Checker
Author: LLM Configurator

Written by Jakub Rusinowski · Last updated July 21, 2026

The reference datacenter GPU for LLM serving: 80 GB HBM3, FP8 transformer engine, first-class vLLM/TensorRT-LLM support. One H100 serves a 70B model to a whole department; 2–4× serve 100B+ MoE models. No official MSRP — figure shown is a typical single-unit PCIe street price; most businesses buy it inside an OEM server or rent it hourly (see the cloud directory) before committing.

Technical Specifications

VRAM	80 GB
Memory Bandwidth	2000 GB/s
TDP	350 W
Architecture	Hopper GH100
Release Year	2022
MSRP at Launch	$25,000
Inference Speed (Llama 3.1 8B Q4_K_M)	120–250 tok/s (estimated)
Inference Speed (Llama 3.3 70B Q4_K_M)	23–48 tok/s (estimated)

Buy This HardwareNVIDIA H100 80GB PCIe — 80 GB VRAM · 350 W board power Deploy in the Cloud NowNVIDIA H100 80GB on RunPod — from $2.89/hr · rate checked 2026-07

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Affiliate disclosure: Some links on this page are affiliate links — if you buy through them, LLM Configurator may earn a commission at no extra cost to you. As an Amazon Associate, LLM Configurator earns from qualifying purchases.

NVIDIA H100 80GB PCIe

80 GB VRAM · 350 W board power

2026 prices are volatile — check the current listing.

Check price on Amazon

LLMs Compatible with 80 GB VRAM

All models below run comfortably in 80 GB VRAM with Q4_K_M quantization.

Llama 4	Llama 4 Scout 17B · 67 GB VRAM · Q4_K_M · `ollama run llama4:scout`
Llama 3.3	Llama 3.3 70B Instruct · 43 GB VRAM · Q4_K_M · `ollama run llama3.3`
Llama 3.1 Family	Llama 3.1 8B Instruct · 6 GB VRAM · Q4_K_M · `ollama run llama3.1`
DeepSeek R1	DeepSeek R1 Distill Qwen 32B · 20 GB VRAM · Q4_K_M · `ollama run deepseek-r1:32b`
Qwen 3	Qwen 3 32B · 21 GB VRAM · Q4_K_M · `ollama run qwen3:32b`
Qwen 3.5	Qwen 3.5 122B-A10B · 74 GB VRAM · Q4_K_M · `ollama run qwen3.5:122b`
Qwen 3.6	Qwen 3.6 35B-A3B · 22 GB VRAM · Q4_K_M · `ollama run qwen3.6:35b-a3b`
Gemma 3	Gemma 3 27B Instruct · 17 GB VRAM · Q4_K_M · `ollama run gemma3:27b`

Best Use Cases

production inference at scale
70B+ models
fine-tuning
vLLM serving

Quick Start with Ollama

Install Ollama then run the recommended model for this GPU:

ollama run llama3.3:70b

FAQ

Can the NVIDIA H100 80GB run local LLMs?

Yes — the NVIDIA H100 80GB has 80 GB VRAM and runs The reference datacenter GPU for LLM serving: 80 GB HBM3, FP8 transformer engine, first-class vLLM/TensorRT-LLM support.

How fast is the NVIDIA H100 80GB for AI inference?

The NVIDIA H100 80GB is estimated to run Llama 3.1 8B at 120–250 tok/s with Q4_K_M quantization. For Llama 3.3 70B the estimate is 23–48 tok/s. These are modelled estimates, not measurements — see /en/methodology.

What LLMs can I run on 80 GB VRAM?

With 80 GB you can run: Llama 4, Llama 3.3, Llama 3.1 Family, DeepSeek R1, Qwen 3. Use Ollama for the easiest setup: ollama run llama3.3:70b.

Can I Run It? — NVIDIA H100 80GB

Compare Similar GPUs

NVIDIA A100 80GB (80 GB, 145 t/s)

VRAM Tier

Best LLMs for 80 GB VRAM

Buying Guide

Best GPU Buyer Guide 2026

← All GPU Reviews | Check Your Hardware | Full Benchmarks | Can I Run It?