NVIDIADesktop GPUCurrent

NVIDIA H100 80GB (PCIe) for local LLMs

Name: LLM Configurator — GPU VRAM Checker
Author: LLM Configurator

Written by Jakub Rusinowski · Last updated July 21, 2026

With 80 GB at 2,000 GB/s, the H100 80GB (PCIe) runs 130 catalogued models at Q4_K_M with 8K context. The largest that fits is Devstral-2 123B (~75.1 GB), and the top pick is GPT-OSS 120B at 147–306 tok/s.

The reference datacenter GPU for LLM serving: 80 GB HBM3, FP8 transformer engine, first-class vLLM/TensorRT-LLM support. One H100 serves a 70B model to a whole department; 2–4× serve 100B+ MoE models. No official MSRP — figure shown is a typical single-unit PCIe street price; most businesses buy it inside an OEM server or rent it hourly (see the cloud directory) before committing.

Check my hardware →Full benchmark table

Models that run on the H100 80GB (PCIe)

Q4_K_M, 8K context, 80 GB usable. Ranked by quality and speed.

Model	VRAM	Speed	Fit
GPT-OSS 120B GPT-OSS	71.9 GB	147–306 tok/s	Fits
Llama 4.5 Scout Llama 4.5	66.6 GB	66–136 tok/s	Fits
Llama 4 Scout 17B Llama 4	68.2 GB	70–146 tok/s	Fits
Llama 3.2 Vision 90B Llama 3.2 Vision	57.8 GB	18–38 tok/s	Fits
Qwen3-Coder-Next (80B-A3B MoE) Qwen3-Coder	49.1 GB	136–283 tok/s	Fits
Kolibri 1 Kolibri	48.4 GB	182–378 tok/s	Fits
Llama 3.3 70B Instruct Llama 3.3	45.7 GB	23–47 tok/s	Fits
Cogito v1 70B Cogito v1	45.7 GB	23–47 tok/s	Fits
Apertus 70B Apertus	43.1 GB	23–47 tok/s	Fits
Cosmos 3 Super Cosmos 3	39.4 GB	43–90 tok/s	Fits
K2 Horizon MoVA 36B-A4B K2 Horizon	25 GB	141–293 tok/s	Fits
Qwen 3.6 35B-A3B Qwen 3.6	22.1 GB	191–396 tok/s	Fits

Showing 12 of 130

Buy it or rent the same memory

Buy the card, or rent a GPU with the same memory by the hour to try models first.

Buy this hardware NVIDIA H100 80GB PCIe — 80 GB VRAM · 350 W board power Deploy in the cloud now NVIDIA H100 80GB on RunPod — from $2.89/hr · rate checked 2026-07

or compare on Vast.ai

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Affiliate disclosure: Some links on this page are affiliate links — if you buy through them, LLM Configurator may earn a commission at no extra cost to you. As an Amazon Associate, LLM Configurator earns from qualifying purchases.

NVIDIA H100 80GB PCIe

80 GB VRAM · 350 W board power

2026 prices are volatile — check the current listing.

Check price on Amazon

Speed vs other GPUs

Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated

H100 80GB (SXM)157–327 tok/s H100 80GB (PCIe)120–250 tok/s RTX 5090113–234 tok/s RTX PRO 6000 Blackwell113–234 tok/s RTX PRO 6000 Blackwell (Max-Q Edition)113–234 tok/s A100 80GB (PCIe)102–196 tok/s RTX PRO 5000 Blackwell (48 GB)94–194 tok/s RTX PRO 5000 Blackwell (72 GB)94–194 tok/s

Specifications

Specs last updated 2026-07-21.

Memory: 80 GB
Memory bandwidth: 2,000 GB/s
Architecture: Hopper GH100
Series: Server
Board power: 350 W
Release year: 2022
Launch price: $25,000
Usable for models: 80 GB

Best forproduction inference at scale70B+ modelsfine-tuningvLLM serving

Recommended first model (Ollama)

Frequently asked questions

Can the NVIDIA H100 80GB (PCIe) run local LLMs?

Yes. With 80 GB (80 GB usable by a model) it runs 130 of the catalogued models at Q4_K_M with 8K context; the largest is Devstral-2 123B, needing about 75.1 GB.

How fast is the NVIDIA H100 80GB (PCIe) for AI inference?

It is estimated to run Llama 3.1 8B at 120–250 tok/s at Q4_K_M. Llama 3.3 70B is estimated at 23–48 tok/s. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.

What LLMs can I run on 80 GB?

Among the best that fit: GPT-OSS 120B, Llama 4.5 Scout, Llama 4 Scout 17B, Llama 3.2 Vision 90B, Qwen3-Coder-Next (80B-A3B MoE). The quickest start is Ollama: ollama run llama3.3:70b.

Can I run it on the H100 80GB (PCIe)?

More for this card

Tools

Guides