Command R+ (104B) — VRAM, Speed & Local Setup

Written by Jakub Rusinowski · Last updated April 4, 2024

Model library → Command R Family → Command R+ (104B)

Cohere's flagship model for complex enterprise RAG. Handles multi-hop retrieval, grounded citations, and complex tool orchestration. Outperforms GPT-4 Turbo on RAG benchmarks.

Command R+ (104B) needs about 64 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache. On Apple Silicon that figure comes out of unified memory.

Specifications

Parameters104 Billion
Context window128,000
ArchitectureDense
ProviderCohere
LicenceCC-BY-NC
Specified atQ4_K_M
System RAM128 GB
Record updated2024-04-04

Licence

CC-BY-NC-4.0 — research / non-commercial only. Research / non-commercial only — this licence does NOT permit shipping a commercial product.

VRAM and Speed by Quantization

Modelled on a reference NVIDIA RTX 4090 (24 GB), at 8K context. Speed figures are ESTIMATES from the memory-bandwidth roofline described on the methodology page, not benchmarks we ran — rows marked measured come from published or reader-submitted runs. VRAM here includes the KV cache, so it reads higher than the headline figure above, which does not.

QuantWeightsVRAM neededEst. speedFit on 24 GB
Q2_K34.2 GB37.1 GB~3 tok/s (est.)Offloads to system RAM (slow)
Q3_K_M44.3 GB47.3 GB~2 tok/s (est.)Offloads to system RAM (slow)
Q4_K_M62.8 GB65.7 GB—Won't fit
Q5_K_M73.7 GB76.7 GB—Won't fit
Q6_K85.3 GB88.2 GB—Won't fit
Q8_0110.5 GB113.4 GB—Won't fit
F16208.0 GB210.9 GB—Won't fit

Want the memory numbers alone, at every quantization level and your own context length? Use the Command R+ (104B) VRAM calculator.

Buy This HardwareApple MacBook Pro M5 Pro — 64 GB VRAM · 30 W board powerDeploy in the Cloud NowNVIDIA A100 80GB on RunPod — from $1.39/hr · rate checked 2026-07

or compare on Vast.ai from $0.77/hr (typical low · varies)

As an Amazon Associate we earn from qualifying purchases. Cloud GPU links are referral links — we may earn a commission at no extra cost to you.

Recommended GPU

The cheapest catalogued GPU that runs Command R+ (104B) is the Apple M5 Pro (64 GB).

Affiliate disclosure: Some links on this page are affiliate links — if you buy through them, LLM Configurator may earn a commission at no extra cost to you. As an Amazon Associate, LLM Configurator earns from qualifying purchases.
Apple MacBook Pro M5 Pro
64 GB VRAM · 30 W board power
2026 prices are volatile — check the current listing.
Check price on Amazon

How to Run Command R+ (104B)

Install Ollama, then run:

ollama run command-r-plus

Weights on Hugging Face: CohereForAI/c4ai-command-r-plus.

Best for: enterprise rag, agent, complex retrieval.

Can I Run Command R+ (104B) on My GPU?

Other Command R Family Sizes

Command R+ (104B) — Frequently Asked Questions

How much VRAM does Command R+ (104B) need?
About 64 GB at Q4_K_M — quantized weights plus framework overhead, before any KV cache. The cache grows with context length and is added on top; the table above folds it in. Apple Silicon counts unified memory toward the same figure.
Does Command R+ (104B) run on an RTX 4090 (24 GB)?
No. Command R+ (104B) needs about 64 GB at Q4_K_M, more than a single RTX 4090's 24 GB. It needs a larger card, several GPUs, or Apple Silicon with enough unified memory — or it runs with part of the weights offloaded to system RAM, which is much slower.
How do I run Command R+ (104B) locally?
Install Ollama and run `ollama run command-r-plus`. That pulls the weights and starts a local OpenAI-compatible endpoint; after the download nothing leaves the machine.
What other sizes does Command R Family come in?
Command R (35B) (22 GB), Command R+ (104B) (64 GB). Every size shares the family's training and licence; the larger ones score higher and need proportionally more memory.

← All Command R Family models | VRAM calculator | Check your own hardware