NVIDIA H100 80GB — Local LLM Performance & Compatibility

Name: LLM Configurator — GPU VRAM Checker
Author: LLM Configurator

The reference datacenter GPU for LLM serving: 80 GB HBM3, FP8 transformer engine, first-class vLLM/TensorRT-LLM support. One H100 serves a 70B model to a whole department; 2–4× serve 100B+ MoE models. No official MSRP — figure shown is a typical single-unit PCIe street price; most businesses buy it inside an OEM server or rent it hourly (see the cloud directory) before committing.

Technical Specifications

VRAM	80 GB
Memory Bandwidth	2000 GB/s
TDP	350 W
Architecture	Hopper GH100
Release Year	2022
MSRP at Launch	$25,000
Inference Speed (Llama 3.1 8B Q4_K_M)	~190 tokens/sec
Inference Speed (Llama 3.3 70B Q4_K_M)	~40 tokens/sec

联盟营销声明: 本页部分链接为联盟推广链接——如果你通过它们购买，LLM Configurator 可能会获得佣金，而你无需支付任何额外费用。作为亚马逊联盟成员（Amazon Associate），LLM Configurator 会从符合条件的购买中获得收益。

NVIDIA H100 80GB PCIe

首发建议零售价：$25,000

2026年价格波动较大——请以当前商品页价格为准。

在亚马逊查看价格

LLMs Compatible with 80 GB VRAM

All models below run comfortably in 80 GB VRAM with Q4_K_M quantization.

Llama 4	67 GB VRAM · Q4_K_M · `ollama run llama4:scout`
Llama 3.3	43 GB VRAM · Q2_K_XS (Tight) · `ollama run llama3.3`
Llama 3.1 Family	6 GB VRAM · Q4_K_M · `ollama run llama3.1`
DeepSeek R1	20 GB VRAM · Q4_K_M · `ollama run deepseek-r1:32b`
Qwen 3	80 GB VRAM · Q4_K_M · `ollama run qwen3:235b-a22b`
Qwen 3.5	74 GB VRAM · Q4_K_M · `ollama run qwen3.5:122b`
Qwen 3.6	22 GB VRAM · Q4_K_M · `ollama run qwen3.6:35b-a3b`
Gemma 3	17 GB VRAM · Q4_K_M · `ollama run gemma3:27b`

Best Use Cases

production inference at scale
70B+ models
fine-tuning
vLLM serving

Quick Start with Ollama

Install Ollama then run the recommended model for this GPU:

ollama run llama3.3:70b

FAQ

Can the NVIDIA H100 80GB run local LLMs?

Yes — the NVIDIA H100 80GB has 80 GB VRAM and runs The reference datacenter GPU for LLM serving: 80 GB HBM3, FP8 transformer engine, first-class vLLM/TensorRT-LLM support.

How fast is the NVIDIA H100 80GB for AI inference?

The NVIDIA H100 80GB runs Llama 3.1 8B at ~190 tokens/sec with Q4_K_M quantization. For the 70B model it achieves ~40 tokens/sec.

What LLMs can I run on 80 GB VRAM?

With 80 GB you can run: Llama 4, Llama 3.3, Llama 3.1 Family, DeepSeek R1, Qwen 3. Use Ollama for the easiest setup: ollama run llama3.3:70b.

Can I Run It? — NVIDIA H100 80GB

Compare Similar GPUs

NVIDIA A100 80GB (80 GB, 145 t/s)

VRAM Tier

Best LLMs for 80 GB VRAM

Buying Guide

Best GPU Buyer Guide 2026

← All GPU Reviews | Check Your Hardware | Full Benchmarks | Can I Run It?