Written by Jakub Rusinowski · Last updated July 21, 2026
The widest hardware choice and the easiest NVIDIA setup, at the cost of the highest idle memory overhead of the three platforms.
| Runtime | How to install | Acceleration | Watch out for |
|---|---|---|---|
| Ollama | Official Windows installer (.exe), or winget.winget install Ollama.Ollama | CUDA (NVIDIA), ROCm (select AMD), Vulkan, CPU | ROCm support on Windows covers far fewer AMD cards than on Linux; check your specific model before relying on it. |
| LM Studio | Official Windows installer, GUI-first with a built-in model browser. | CUDA (NVIDIA), Vulkan, CPU | — |
| llama.cpp | Prebuilt release binaries, or build from source with CUDA enabled.winget install llama.cpp | CUDA (NVIDIA), Vulkan, SYCL (Intel), CPU | Prebuilt Windows binaries are split by backend — download the CUDA build, not the CPU one, or you will silently run on CPU. |
| vLLM | Runs under WSL2, not natively on Windows.wsl --install | CUDA (NVIDIA, via WSL2) | WSL2 splits memory with Windows and adds a virtualisation layer; for single-user local inference Ollama or llama.cpp is the simpler choice. |
Windows supports NVIDIA, AMD, Intel accelerators — 37 of the 55 in our hardware database, up to 128 GB.
| Memory | Hardware on this platform | Models to run |
|---|---|---|
| 8 GB | NVIDIA GeForce RTX 3080 (10GB) Intel Arc B570 NVIDIA GeForce RTX 5060 Ti 8GB | Qwen 3 8B GLM-6 9B GLM-4.7 9B |
| 16 GB | AMD Radeon RX 7900 XT NVIDIA GeForce RTX 5080 NVIDIA GeForce RTX 5070 Ti | Mistral Small 3.1 24B Qwen 3.5 14B Qwen 3.5 14B |
| 24 GB | NVIDIA GeForce RTX 4090 NVIDIA GeForce RTX 3090 AMD Radeon RX 7900 XTX | Gemma 4 27B ⭐ Mistral Small 3.1 24B Qwen 3.7 35B-A3B |
| 48 GB | NVIDIA RTX 6000 Ada Generation NVIDIA L40S | Nemotron 70B Instruct GLM-5.1 72B Qwen 3.5 72B |
| 96 GB | AMD Ryzen AI Max+ 395 NVIDIA RTX PRO 6000 Blackwell | GPT-oss 120B Nemotron 70B Instruct Qwen 3.5 122B-A10B (MoE) |
| Machine | Memory for models | Form factor | Upgradeable |
|---|---|---|---|
| RTX 4090 Desktop (24 GB VRAM, 64 GB RAM) | 24 GB VRAM | Desktop | Yes |
| RTX 5090 Desktop (32 GB VRAM, 64 GB RAM) | 32 GB VRAM | Desktop | Yes |
| RTX 3090 Desktop (24 GB VRAM, 64 GB RAM) | 24 GB VRAM | Desktop | Yes |
| RTX 5080 Desktop (16 GB VRAM, 32 GB RAM) | 16 GB VRAM | Desktop | Yes |
| RTX 3060 12 GB Desktop (12 GB VRAM, 32 GB RAM) | 12 GB VRAM | Desktop | Yes |
| RTX 5060 Ti 16 GB Desktop (16 GB VRAM, 32 GB RAM) | 16 GB VRAM | Desktop | Yes |
| Framework Desktop (Ryzen AI Max+ 395, 128 GB) | 128 GB unified | Mini PC | No |
| RTX 4060 Laptop (8 GB VRAM, 16 GB RAM) | 8 GB VRAM | Laptop | Yes |
Ollama — Official Windows installer (.exe), or winget. It reaches CUDA (NVIDIA), ROCm (select AMD), Vulkan, CPU.
NVIDIA, AMD, Intel hardware, up to 128 GB in our database.
Windows itself reserves noticeably more VRAM and system RAM than Linux, so usable memory for a model is lower on identical hardware.