Aya 3B (Tiny Aya) — local AI model by Cohere
Written by Jakub Rusinowski · Last updated
Cohere's February 17, 2026 release — the most capable multilingual open-weight model at the 3B parameter class. Tiny Aya outperforms Qwen3-4B, Gemma 3 4B, and Ministral 3B on multilingual benchmarks across 23 languages. Perfect for edge devices, phones, and resource-constrained deployments where global language support is critical.
Variants
The smallest Aya 3B (Tiny Aya) variant needs about 3 GB of VRAM at Q4_K_M — quantized weights plus framework overhead, before any KV cache.
| Model | VRAM at Q4 | VRAM | Context | Run it |
|---|---|---|---|---|
| Aya 3B → 3B | ~2.6 GB | 8,192 | ollama run aya:3b |
Memory is quantized weights plus overhead at Q4_K_M, from the same engine as the GPU & VRAM checker.
How to run Aya 3B (Tiny Aya) locally
Install Ollama, then pull the tag.
ollama run aya:3bPick a size above for its own VRAM figure, speed estimate and install command.
Licence
Research / non-commercial only — this licence does NOT permit shipping a commercial product.
Applies to: Aya 3BRecommended GPU
The cheapest catalogued GPU that runs Aya 3B (Tiny Aya) locally (min 3 GB VRAM) is the Intel Arc B570 (10 GB).
Can I run Aya 3B (Tiny Aya) on my GPU?
Aya 3B (Tiny Aya) — frequently asked questions
How much VRAM does Aya 3B (Tiny Aya) need?
Aya 3B (Tiny Aya) needs about 3 GB VRAM at Q4_K_M quantization for its smallest variant. Variants: Aya 3B (3 GB, Q4_K_M). On Apple Silicon, unified memory counts toward this requirement.
Can I run Aya 3B (Tiny Aya) on an RTX 4090 (24 GB)?
Yes — Aya 3B (Tiny Aya) runs on an RTX 4090 (24 GB) and other 24 GB cards such as the RTX 3090. Smaller variants also fit comfortably on 8–16 GB GPUs at Q4_K_M.
What quantization should I use for Aya 3B (Tiny Aya)?
Q4_K_M is the best balance of quality and VRAM for Aya 3B (Tiny Aya) in most cases. Choose Q8_0 for near-lossless quality if you have spare VRAM, or smaller quants (Q3/Q2) only when memory is tight.
How do I run Aya 3B (Tiny Aya) with Ollama?
Install Ollama, then run: ollama run aya:3b. This downloads Aya 3B (Tiny Aya) and starts a local, OpenAI-compatible endpoint — no internet connection is needed after the initial download.