Apple M2 Ultra for local LLMs
Written by Jakub Rusinowski · Last updated
With 192 GB at 800 GB/s, the M2 Ultra runs 143 catalogued models at Q4_K_M with 8K context. The largest that fits is MiniMax M2.7 230B-A10B (~139.7 GB), and the top pick is Qwen3.8-Flash-Next at 55–115 tok/s.
Up to 192 GB unified memory at 800 GB/s — among the largest memory pools available on consumer hardware. Runs 70B models with room to spare. Available in Mac Studio and Mac Pro.
Models that run on the M2 Ultra
Q4_K_M, 8K context, 144 GB usable. Ranked by quality and speed.
| Model | Memory · marker = 144 GB | VRAM | Speed |
|---|---|---|---|
| Qwen3.8-Flash-Next Qwen3.8 | 109.7 GB | 55–115 tok/s | |
| Qwen 3.5 122B-A10B Qwen 3.5 | 74.5 GB | 28–58 tok/s | |
| Nemotron 3 Super 120B-A12B Nemotron 3 Super | 73.3 GB | 25–52 tok/s | |
| Nemotron 3.5 Super VL 120B-A12B Nemotron 3.5 | 73.3 GB | 25–52 tok/s | |
| Mistral Small 4 119B-A6.5B Mistral Small 4 | 72.6 GB | 16–34 tok/s | |
| GPT-OSS 120B GPT-OSS | 71.6 GB | 60–125 tok/s | |
| Llama 4.5 Scout Llama 4.5 | 66.6 GB | 20–41 tok/s | |
| Llama 4 Scout 17B Llama 4 | 68.2 GB | 22–45 tok/s | |
| Qwen3-Coder-Next (80B-A3B MoE) Qwen3-Coder | 49.4 GB | 84–174 tok/s | |
| Kolibri 1 Kolibri | 48.5 GB | 78–161 tok/s | |
| Nemotron 3.5 Bolt 74B Nemotron 3.5 | 45.6 GB | 42–87 tok/s | |
| Cosmos 3 Super Cosmos 3 | 39.4 GB | 12–26 tok/s |
Buy it or rent the same memory
Buy the card, or rent a GPU with the same memory by the hour to try models first.
As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.
Speed vs other GPUs
Llama 3.1 8B, Q4_K_M. Estimated ranges. How this is calculated
Specifications
Specs last updated 2026-07-12.
- Memory
- 192 GB
- Memory bandwidth
- 800 GB/s
- Architecture
- ARM, 5nm TSMC
- Series
- Apple Silicon
- Board power
- 60 W
- Release year
- 2023
- Usable for models
- 144 GB (~75% of unified)
Similar GPUs
Frequently asked questions
Can the Apple M2 Ultra run local LLMs?
Yes. With 192 GB (144 GB usable by a model) it runs 143 of the catalogued models at Q4_K_M with 8K context; the largest is MiniMax M2.7 230B-A10B, needing about 139.7 GB.
How fast is the Apple M2 Ultra for AI inference?
It is estimated to run Llama 3.1 8B at 43–89 tok/s at Q4_K_M. Llama 3.3 70B is estimated at 6.3–13 tok/s. These are modelled estimates from memory bandwidth, not measurements; the methodology page shows the formula.
What LLMs can I run on 192 GB?
Among the best that fit: Qwen3.8-Flash-Next, Qwen 3.5 122B-A10B, Nemotron 3 Super 120B-A12B, Nemotron 3.5 Super VL 120B-A12B, Mistral Small 4 119B-A6.5B. The quickest start is Ollama: ollama run llama3.3:70b.
Keep going
Can I run it on the M2 Ultra?
More for this card
Tools