Best local LLMs for your GPU

Search your card, laptop GPU or Mac. We rank everything that runs on it by quality and speed, and show what gets cut if you need more context.

Your hardware

RTX 3060 (12GB)RTX 4060 Ti 16GBRTX 3090RTX 4090RTX 5090M4 Max

Context length8K tokens

4K8K16K32K128K

Ranked for the RTX 4090 at 8K context. Pick your own card above once the page has loaded.

RTX 409024 GBusable · 1008 GB/s

Models that run1113 more with RAM offload

Best overall87DeepSeek R1 Distill Qwen 32B

Fastest good model~237 tok/sLFM2.5-8B-A1B

All tasksCodingChatReasoningAgentsVisionRuns on GPURAM offloadEverything

#	Model	VRAM	Speed (est.)	Score	Fit
1	DeepSeek R1 Distill Qwen 32B DeepSeek R1 · 32B	22.3 GB	~34 tok/s	87	Tight
2	DeepSeek R1 Distill Qwen 14B DeepSeek R1 · 14B	10.6 GB	~70 tok/s	80	Fits
3	Qwen 3.5 27B Qwen 3.5 · 27B	17.1 GB	~40 tok/s	79	Fits
4	Laguna XS 2.1 33B-A3B Poolside Laguna XS 2.1 · 33B (3B active)	20.7 GB	~171 tok/s	79	Fits
5	Granite 4.0 Small-H 32B-A9B IBM Granite 4.0 · 32B (9B active)	20.1 GB	~94 tok/s	79	Fits
6	Olmo 3 32B Olmo 3 · 32B	20.1 GB	~34 tok/s	79	Fits
7	Nemotron-Cascade 2 30B-A3B Nemotron Cascade 2 · 31.6B (3.5B active)	19.9 GB	~160 tok/s	79	Fits
8	Gemma 4 31B Gemma 4 · 31B	19.5 GB	~35 tok/s	79	Fits
9	Qwen 3 30B-A3B (MoE) Qwen 3 · 30B (3B active)	20 GB	~185 tok/s	79	Fits
10	Qwen3-Coder 30B-A3B (MoE) Qwen3-Coder · 30.5B (3.3B active)	20 GB	~185 tok/s	79	Fits
11	GLM-4.7-Flash 30B-A3B GLM-4.7 / GLM-Z1 · 30B (3B active)	18.9 GB	~172 tok/s	79	Fits
12	Granite 4.1 30B IBM Granite 4.1 · 30B	18.9 GB	~37 tok/s	79	Fits
13	Nemotron 3 Nano Omni 30B-A3B Nemotron 3 Nano Omni · 30B/3B active	18.9 GB	~172 tok/s	79	Fits
14	North Mini Code 1.0 30B-A3B North Mini Code · 30B (3B active)	18.9 GB	~172 tok/s	79	Fits
15	Granite 4.2 30B IBM Granite 4.2 · 30B	18.9 GB	~37 tok/s	79	Fits
16	Nemotron 3.5 Lightning 30B-A3B Nemotron 3.5 · 30B (3B active)	18.9 GB	~172 tok/s	79	Fits
17	Muse Glimmer 30B Muse Glimmer · 29.6B	18.7 GB	~37 tok/s	79	Fits
18	Qwen 3.6 27B Qwen 3.6 · 27.8B	17.6 GB	~39 tok/s	79	Fits
19	Qwen3.8 27B Qwen3.8 · 27B	18.1 GB	~40 tok/s	79	Fits
20	1-bit Bonsai 27B Bonsai 27B · 27B	17.1 GB	~40 tok/s	79	Fits
21	Ternary Bonsai 27B Bonsai 27B · 27B	17.1 GB	~40 tok/s	79	Fits
22	Trinity Mini Trinity · 26B (3B active)	16.9 GB	~211 tok/s	79	Fits
23	Gemma 4 26B-A4B Gemma 4 · 26B (4B active, MoE)	16.5 GB	~152 tok/s	79	Fits
24	Devstral Small 2505 24B Devstral · 24B	16.6 GB	~45 tok/s	79	Fits
25	Devstral Small 2 24B Devstral · 24B	16.6 GB	~45 tok/s	79	Fits

Showing 25 of 111 · Q4_K_M, 8K context

A page for every card

The strongest models per workload, the largest model each card holds, and what it cannot run.

NVIDIA

Apple

Intel

Intel Arc Pro B70 (32 GB)Intel Arc Pro B65 (32 GB)Intel Arc B580 (12 GB)Intel Arc B570 (10 GB)Intel Arc Pro B60 (24 GB)Intel Arc Pro B50 (16 GB)

AMD