Apple M3 for local LLMs
Written by Jakub Rusinowski · Last updated
With 24 GB at 100 GB/s, the M3 runs 92 catalogued models at Q4_K_M with 8K context. The largest that fits is Ternary Bonsai 2 27B (~17.6 GB), and the top pick is LFM2.5-24B-A2B at 20–41 tok/s.
Up to 24 GB unified memory at 100 GB/s — same memory ceiling as M2 but on the 3nm process. Fits 7–8B models at Q4 comfortably. Common in the MacBook Air, iMac, and Mac mini.
Models that run on the M3
Q4_K_M, 8K context, 18 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 18 GB | VRAM | Speed |
|---|---|---|---|
| LFM2.5-24B-A2B LFM2.5 | 15.5 GB | 20–41 tok/s | |
| GPT-OSS 20B GPT-OSS | 13.6 GB | 14–29 tok/s | |
| Mellum2.1 12B Thinking Mellum2.1 | 9 GB | 20–41 tok/s | |
| Gemma 3n E2B Gemma 3n | 4.1 GB | 15–32 tok/s | |
| LFM2.5-8B-A1B LFM2.5 | 5.8 GB | 16–33 tok/s | |
| Trinity Mini Trinity | 16.9 GB | 17–35 tok/s | |
| Ternary Bonsai 2 4B Ternary Bonsai 2 | 3.6 GB | 12–25 tok/s | |
| Gemma 4 26B-A4B Gemma 4 | 17 GB | 12–24 tok/s | |
| Gemma 3n E4B Gemma 3n | 5.5 GB | 9.8–20 tok/s | |
| Llama 3.2 1B Instruct Llama 3.2 Family | 1.8 GB | 30–63 tok/s | |
| Llama 3.2 3B Instruct Llama 3.2 Family | 3.7 GB | 12–24 tok/s | |
| StarCoder 2 3B StarCoder 2 | 2.9 GB | 16–33 tok/s |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
or compare on Vast.ai from $0.35/hr (typical low · varies)
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-07-12.
- Memory
- 24 GB
- Memory bandwidth
- 100 GB/s
- Architecture
- ARM, 3nm TSMC
- Series
- Apple Silicon
- Board power
- 22 W
- Release year
- 2023
- Usable for models
- 18 GB (~75% of unified)
Similar GPUs
Frequently asked questions
Can the Apple M3 run local LLMs?
Yes. With 24 GB (18 GB usable by a model) it runs 92 of the catalogued models at Q4_K_M with 8K context; the largest is Ternary Bonsai 2 27B, needing about 17.6 GB.
How fast is the Apple M3 for AI inference?
It is estimated to run Llama 3.1 8B at 6.4–13 tok/s at Q4_K_M. Llama 3.3 70B does not fit: it needs about 44 GB against 18 GB usable. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.
What LLMs can I run on 24 GB?
Among the best that fit: LFM2.5-24B-A2B, GPT-OSS 20B, Mellum2.1 12B Thinking, Gemma 3n E2B, LFM2.5-8B-A1B. The quickest start is Ollama: ollama run llama3.2:3b.
Keep going
Can I run it on the M3?
More for this card
Tools