Written by Jakub Rusinowski · Last updated July 21, 2026
The fastest and most memory-efficient platform, and the only one where every serving stack runs natively — at the cost of doing your own driver work.
| Runtime | How to install | Acceleration | Watch out for |
|---|---|---|---|
| Ollama | Official install script.curl -fsSL https://ollama.com/install.sh | sh | CUDA (NVIDIA), ROCm (AMD), Vulkan, CPU | — |
| llama.cpp | Build from source with the backend flag for your GPU.cmake -B build -DGGML_CUDA=ON && cmake --build build --config Release | CUDA (NVIDIA), ROCm (AMD), Vulkan, SYCL (Intel), CPU | Build flags decide the backend — a default build is CPU-only and will be an order of magnitude slower than it should be. |
| vLLM | pip, ideally into a clean virtual environment.pip install vllm | CUDA (NVIDIA), ROCm (AMD) | Built for concurrent serving, and it pre-allocates most of the GPU's memory on startup. For a single user it uses far more memory than llama.cpp for no benefit. |
| LM Studio | AppImage. | CUDA (NVIDIA), Vulkan, CPU | — |
Linux supports NVIDIA, AMD, Intel accelerators — 37 of the 55 in our hardware database, up to 128 GB.
| Memory | Hardware on this platform | Models to run |
|---|---|---|
| 8 GB | NVIDIA GeForce RTX 3080 (10GB) Intel Arc B570 NVIDIA GeForce RTX 5060 Ti 8GB | Qwen 3 8B GLM-6 9B GLM-4.7 9B |
| 16 GB | AMD Radeon RX 7900 XT NVIDIA GeForce RTX 5080 NVIDIA GeForce RTX 5070 Ti | Mistral Small 3.1 24B Qwen 3.5 14B Qwen 3.5 14B |
| 24 GB | NVIDIA GeForce RTX 4090 NVIDIA GeForce RTX 3090 AMD Radeon RX 7900 XTX | Gemma 4 27B ⭐ Mistral Small 3.1 24B Qwen 3.7 35B-A3B |
| 48 GB | NVIDIA RTX 6000 Ada Generation NVIDIA L40S | Nemotron 70B Instruct GLM-5.1 72B Qwen 3.5 72B |
| 96 GB | AMD Ryzen AI Max+ 395 NVIDIA RTX PRO 6000 Blackwell | GPT-oss 120B Nemotron 70B Instruct Qwen 3.5 122B-A10B (MoE) |
| Machine | Memory for models | Form factor | Upgradeable |
|---|---|---|---|
| RTX 4090 Desktop (24 GB VRAM, 64 GB RAM) | 24 GB VRAM | Desktop | Yes |
| RTX 5090 Desktop (32 GB VRAM, 64 GB RAM) | 32 GB VRAM | Desktop | Yes |
| RTX 3090 Desktop (24 GB VRAM, 64 GB RAM) | 24 GB VRAM | Desktop | Yes |
| RTX 5080 Desktop (16 GB VRAM, 32 GB RAM) | 16 GB VRAM | Desktop | Yes |
| RTX 3060 12 GB Desktop (12 GB VRAM, 32 GB RAM) | 12 GB VRAM | Desktop | Yes |
| RTX 5060 Ti 16 GB Desktop (16 GB VRAM, 32 GB RAM) | 16 GB VRAM | Desktop | Yes |
| Framework Desktop (Ryzen AI Max+ 395, 128 GB) | 128 GB unified | Mini PC | No |
| RTX 4060 Laptop (8 GB VRAM, 16 GB RAM) | 8 GB VRAM | Laptop | Yes |
Ollama — Official install script. It reaches CUDA (NVIDIA), ROCm (AMD), Vulkan, CPU.
NVIDIA, AMD, Intel hardware, up to 128 GB in our database.
NVIDIA driver and CUDA toolkit installation is on you, and a version mismatch between driver, toolkit and runtime is the most common cause of a silent fall back to CPU.