NVIDIA GeForce RTX 3060 Ti for local LLMs
Written by Jakub Rusinowski · Last updated
With 8 GB of GDDR6 at 448 GB/s, the RTX 3060 Ti runs 53 catalogued models at Q4_K_M with 8K context. The largest that fits is Bielik PL 11B v3.0 Instruct (~7.4 GB), and the top pick is Qwen 3.5 9B at 26–50 tok/s.
8 GB of GDDR6 on a 256-bit bus at 14 Gbps: 448 GB/s, which is more bandwidth than the 12 GB RTX 3060 (360 GB/s) but less memory. NVIDIA also sold an 8 GB GDDR6X version; this record is the GDDR6 card. There is no 12 GB RTX 3060 Ti. Ampere: FP16 and BF16 tensor cores, no FP8.
Models that run on the RTX 3060 Ti
Q4_K_M, 8K context, 8 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 8 GB | VRAM | Speed |
|---|---|---|---|
| Qwen 3.5 9B Qwen 3.5 | 6.2 GB | 26–50 tok/s | |
| GLM-4 9B GLM-4.7 / GLM-Z1 | 6.2 GB | 26–50 tok/s | |
| GLM-4.6V-Flash 9B GLM-4.6V | 6.2 GB | 26–50 tok/s | |
| EuroLLM 9B EuroLLM | 6.2 GB | 26–50 tok/s | |
| InternLM 3 8B Instruct InternLM 3 | 6.5 GB | 30–58 tok/s | |
| LFM2.5-8B-A1B LFM2.5 | 5.8 GB | 71–136 tok/s | |
| Qwen 3 8B Qwen 3 | 7 GB | 28–54 tok/s | |
| Aya Expanse 8B Aya Expanse | 6.7 GB | 29–56 tok/s | |
| Ministral 8B Ministral | 6.9 GB | 29–55 tok/s | |
| Gemma 4 E4B Gemma 4 | 5.6 GB | 29–56 tok/s | |
| Cogito v1 8B Cogito v1 | 6.7 GB | 29–56 tok/s | |
| Granite 4.1 8B IBM Granite 4.1 | 5.6 GB | 29–56 tok/s |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-10-07.
- Memory
- 8 GB GDDR6
- Memory bandwidth
- 448 GB/s
- Memory bus
- 256-bit
- Architecture
- Ampere GA104
- Series
- RTX 30-series
- Board power
- 200 W
- Release year
- 2020
- Launch price
- $399
- Compute backends
- CUDA
- Usable for models
- 8 GB
Similar GPUs
Frequently asked questions
Can the NVIDIA GeForce RTX 3060 Ti run local LLMs?
Yes. With 8 GB (8 GB usable by a model) it runs 53 of the catalogued models at Q4_K_M with 8K context; the largest is Bielik PL 11B v3.0 Instruct, needing about 7.4 GB.
How fast is the NVIDIA GeForce RTX 3060 Ti for AI inference?
It is estimated to run Llama 3.1 8B at 32–61 tok/s at Q4_K_M. Llama 3.3 70B does not fit: it needs about 44 GB against 8 GB usable. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.
What LLMs can I run on 8 GB?
Among the best that fit: Qwen 3.5 9B, GLM-4 9B, GLM-4.6V-Flash 9B, EuroLLM 9B, InternLM 3 8B Instruct.