Decode speed for all 153 runnable local LLM variants across 55 GPUs. Per-token latency is modelled as three terms — streaming the active weights and KV cache from memory, any penalty for weights that spill to system RAM, and a fixed framework floor — with efficiency fitted per GPU architecture from 14 published llama-bench measurements (mean absolute error 7.68%). Fastest models on an NVIDIA RTX 4090 (24 GB) at 4K context:
| SmolLM2 (360M, Q4_K_M) | 257–495 tok/s (modelled range) |
| Qwen 3.5 (0.8B, Q4_K_M) | 224–431 tok/s (modelled range) |
| Llama 3.2 Family (1B, Q4_K_M) | 215–412 tok/s (modelled range) |
| Gemma 3 (1B, Q4_K_M) | 213–409 tok/s (modelled range) |
| BitNet b1.58 (3B, 1.58-bit) | 201–385 tok/s (modelled range) |
| Qwen 3.5 (2B, Q4_K_M) | 174–334 tok/s (modelled range) |
| Gemma 3n (2B, Q4_K_M) | 170–327 tok/s (modelled range) |
| EXAONE 3.5 (2.4B, Q4_K_M) | 166–318 tok/s (modelled range) |
| SmolLM2 (1.7B, Q4_K_M) | 163–312 tok/s (modelled range) |
| StarCoder 2 (3B, Q4_K_M) | 159–305 tok/s (modelled range) |
| Aya 3B (Tiny Aya) (3B, Q4_K_M) | 148–284 tok/s (modelled range) |
| IBM Granite 4.1 (3B, Q4_K_M) | 148–284 tok/s (modelled range) |
| Cogito v1 (3B, Q4_K_M) | 146–281 tok/s (modelled range) |
| Falcon 3 (3B, Q4_K_M) | 145–278 tok/s (modelled range) |
| Qwen 3 (30B (3B active), Q4_K_M) | 142–274 tok/s (modelled range) |
| Llama 3.2 Family (3B, Q4_K_M) | 142–273 tok/s (modelled range) |
| Ministral (3B, Q4_K_M) | 136–261 tok/s (modelled range) |
| Phi 3.5 Family (3.8B, Q4_K_M) | 133–256 tok/s (modelled range) |
| GLM-4.7 / GLM-Z1 (30B (3B active), Q4_K_M) | 133–256 tok/s (modelled range) |
| Poolside Laguna XS 2.1 (33B (3B active), Q4_K_M) | 132–254 tok/s (modelled range) |
Modelled figures are shown as ranges rather than single numbers, because the model's accuracy does not justify more precision than that. Rows marked measured come from published benchmarks credited to their publisher with a source URL, or from reader submissions once several runs agree. Speed depends on context length: the same 8B model loses roughly half its throughput between 4K and 65K context, because the KV cache re-read on every token grows until it dwarfs the weights. Mixture-of-Experts models are sized by their TOTAL parameters for the memory-fit check even though only the active experts affect speed, so a 109B model with 17B active will not load on a 24 GB card. The open dataset is published under CC BY 4.0 at /measured-benchmarks.json.