NVIDIA GeForce RTX 3090 Ti — Local LLM Performance & Compatibility
作者： Jakub Rusinowski · 最后更新： 2026年9月19日
最快的Ampere消费级显卡：24 GB、1008 GB/s，在相同容量下与RTX 4090带宽持平。二手购入是获得24 GB最便宜的途径之一，代价是450 W的整卡功耗和缺少FP8。
Technical Specifications
| VRAM | 24 GB |
| Memory Bandwidth | 1008 GB/s |
| TDP | 450 W |
| Architecture | Ampere GA102 |
| Release Year | 2022 |
| MSRP at Launch | $1,999 |
| Inference Speed (Llama 3.1 8B Q4_K_M) | 63–122 tok/s (estimated) |
| Inference Speed (Llama 3.3 70B Q4_K_M) | Does not fit — needs ~44 GB of 24 GB usable |
或在 Vast.ai 比较，低至 $0.35/小时 (typical low · varies)
作为亚马逊联盟成员，我们从符合条件的购买中获得收入。云 GPU 链接为推荐链接——我们可能获得佣金，您无需额外付费。
LLMs Compatible with 24 GB VRAM
All models below run comfortably in 24 GB VRAM with Q4_K_M quantization.
| Command R Family | Command R (35B) · 22 GB VRAM · Q4_K_M · ollama run command-r |
| Qwen 3.5 | Qwen 3.5 35B-A3B · 22 GB VRAM · Q4_K_M · ollama run qwen3.5:35b-a3b |
| Qwen 3.6 | Qwen 3.6 35B-A3B · 22 GB VRAM · Q4_K_M · ollama run qwen3.6:35b-a3b |
| Nex-N2 | Nex-N2 mini · 22 GB VRAM · Q4_K_M · nex-n2 |
| Yi 1.5 Family | Yi 1.5 34B Chat · 22 GB VRAM · Q4_K_M · ollama run yi:34b |
| Qwen 3 | Qwen 3 32B · 21 GB VRAM · Q4_K_M · ollama run qwen3:32b |
| Aya Expanse | Aya Expanse 32B · 20 GB VRAM · Q4_K_M · ollama run aya-expanse:32b |
| DeepSeek R1 | DeepSeek R1 Distill Qwen 32B · 20 GB VRAM · Q4_K_M · ollama run deepseek-r1:32b |
56 more families also fit 24 GB — browse the full model library.
Best Use Cases
- used 24 GB
- value inference
- multi-GPU builds
FAQ
Can the NVIDIA GeForce RTX 3090 Ti run local LLMs?
Yes — the NVIDIA GeForce RTX 3090 Ti has 24 GB VRAM and runs 最快的Ampere消费级显卡：24 GB、1008 GB/s，在相同容量下与RTX 4090带宽持平。二手购入是获得24 GB最便宜的途径之一，代价是450 W的整卡功耗和缺少FP8。
How fast is the NVIDIA GeForce RTX 3090 Ti for AI inference?
The NVIDIA GeForce RTX 3090 Ti is estimated to run Llama 3.1 8B at 63–122 tok/s with Q4_K_M quantization. Llama 3.3 70B does not fit: it needs about 44 GB against 24 GB usable. These are modelled estimates, not measurements — see /en/methodology.
What LLMs can I run on 24 GB VRAM?
With 24 GB you can run: Command R Family, Qwen 3.5, Qwen 3.6, Nex-N2, Yi 1.5 Family. Use Ollama for the easiest setup: ollama run llama3.1:8b.
Compare Similar GPUs
- NVIDIA GeForce RTX 5090 Laptop GPU (24 GB, 0 t/s)
- Intel Arc Pro B70 (32 GB, 0 t/s)
- Intel Arc Pro B65 (32 GB, 0 t/s)
- AMD Radeon AI PRO R9700 (32 GB, 0 t/s)
VRAM Tier
Buying Guide
← All GPU Reviews | All Hardware | Check Your Hardware | Full Benchmarks | Can I Run It?