Llama 3.1 Family — local AI model by Meta
Written by Jakub Rusinowski · Last updated
The default first install and still the most-pulled model on Ollama by a wide margin. Llama 3.1 is legacy-but-safe rather than current-best: newer small-dense models beat it on coding and reasoning at the same size, but nothing matches it for tooling support, community familiarity, and being the model every guide assumes. Keep it as the smoke test, not the recommendation.
Variants
The smallest Llama 3.1 Family variant needs about 6 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache.
| Model | VRAM at Q4 | VRAM | Context | Run it |
|---|---|---|---|---|
| Llama 3.1 8B Instruct → 8B | ~5.6 GB | 128,000 | ollama run llama3.1 |
Memory is quantized weights plus overhead at Q4_K_M, from the same engine as the GPU & VRAM checker.
How to run Llama 3.1 Family locally
Install Ollama, then pull the tag.
ollama run llama3.1Pick a size above for its own VRAM figure, speed estimate and install command.
Download Llama 3.1 Family — GGUF Quantizations
Pick a quantization and open it in LM Studio, Ollama, or Jan, or download the raw .gguf file directly. Quant list and sizes resolved from Hugging Face.
Llama 3.1 8B Instruct — GGUF quants · bartowski/Meta-Llama-3.1-8B-Instruct-GGUF
| Quant | Size | Download (.gguf) |
|---|---|---|
| Q3_K_M | 3.41 GB (est.) | Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf |
| Q4_K_M | 4.83 GB (est.) | Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf |
| Q5_K_M | 5.67 GB (est.) | Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf |
| Q6_K | 6.56 GB (est.) | Meta-Llama-3.1-8B-Instruct-Q6_K.gguf |
| Q8_0 | 8.50 GB (est.) | Meta-Llama-3.1-8B-Instruct-Q8_0.gguf |
Download in LM Studio: lms get bartowski/Meta-Llama-3.1-8B-Instruct-GGUF
Want this model on your phone? You can run it on your desktop with LM Studio and chat from your iPhone or iPad over an encrypted link — see Run LM Studio Models on Your Phone (LM Link).
Licence
Weights are downloadable and commercial use is permitted, subject to the licence’s acceptable-use terms.
Applies to: Llama 3.1 8B InstructRecommended GPU
The cheapest catalogued GPU that runs Llama 3.1 Family locally (min 6 GB VRAM) is the Intel Arc B570 (10 GB).
Can I run Llama 3.1 Family on my GPU?
- Llama 3.1 Family on AMD Radeon RX 9060 XT 8GB
- Llama 3.1 Family on Intel Arc B570
- Llama 3.1 Family on Intel Arc B580
- Llama 3.1 Family on NVIDIA GeForce RTX 3060 (12GB)
- Llama 3.1 Family on NVIDIA GeForce RTX 3070
- Llama 3.1 Family on NVIDIA GeForce RTX 3070 Ti
- Llama 3.1 Family on NVIDIA GeForce RTX 3080 (10GB)
- Llama 3.1 Family on NVIDIA GeForce RTX 4060
- Llama 3.1 Family on NVIDIA GeForce RTX 4070
- Llama 3.1 Family on NVIDIA GeForce RTX 4070 Super
- Llama 3.1 Family on NVIDIA GeForce RTX 4070 Ti
- Llama 3.1 Family on NVIDIA GeForce RTX 5060
- Llama 3.1 Family on NVIDIA GeForce RTX 5060 Ti 8GB
- Llama 3.1 Family on NVIDIA GeForce RTX 5070
Llama 3.1 Family — frequently asked questions
How much VRAM does Llama 3.1 Family need?
Llama 3.1 Family needs about 6 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: Llama 3.1 8B Instruct (6 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run Llama 3.1 Family on an RTX 4090 (24 GB)?
Yes — Llama 3.1 Family runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
What quantization should I use for Llama 3.1 Family?
Q4_K_M is the best balance of quality and VRAM for Llama 3.1 Family in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
How do I run Llama 3.1 Family with Ollama?
Install Ollama, then run: ollama run llama3.1. This downloads Llama 3.1 Family and starts a local, OpenAI-compatible endpoint — no internet connection is needed after the initial download.