Local voice AI in 2026: the open models that listen and talk on your own hardware
Run speech-to-text, text-to-speech and full-duplex voice models offline: which open models run on a CPU or a GPU, and which licences let you ship them.
Every part of a voice assistant now runs on hardware you own. A 1.7-billion-parameter model transcribes English with a 5.40% word error rate, an 82-million-parameter model reads text aloud on a laptop CPU, and a 9B model listens and talks at the same time on a desktop GPU. The catch is not quality any more. It's licences: the three best-rated open voices on the leading blind-listening board come with three different ones. This guide sorts the open voice models by what they do and what they need, and tells you which ones you can put in a product.
If you just want a working setup, jump to the recipes by hardware or open the voice model hub.
The stack below shows one model per stage for each kind of machine, with the licence on each model's weights next to its name.
Key takeaways
- Three ways to build local voice. Chain three models (speech-to-text, then an LLM, then text-to-speech), use a speech-in LLM that hears you but answers in text, or run one full-duplex model that listens and speaks at once. The chain is still the practical default.
- Speech-to-text is solved for most languages. Qwen3-ASR 1.7B leads the open models on the Open ASR Leaderboard, with IBM's new Granite Speech 5.0 and Cohere Transcribe close behind. Whisper is still the one with the most runtimes.
- Text-to-speech quality caught up; licences split three ways. On the Artificial Analysis Speech Arena the top open voice allows commercial use only below a revenue cap, the second is non-commercial, and the third is MIT. Kokoro, Qwen3-TTS, VoxCPM2, Chatterbox and Ming-3.0-tts are the ones you can ship without conditions.
- You don't need a GPU to start. Kokoro, Piper, Pocket TTS, Moonshine and Fun-ASR-Nano all run on a CPU.
- The runtimes moved this week. Ollama added its first text-to-speech model on 24 October, Mac only, and llama.cpp's server now speaks text-to-speech and streaming transcription.
- Full-duplex is here, but mostly on servers. NVIDIA's NemotronLabs VoiceChat 11B is the first open full-duplex model that calls tools mid-conversation; MiniCPM-o 4.5 is the one that fits a desktop.
Three ways to build a local voice assistant
1. The chain (cascade). A speech-to-text model turns audio into text, your usual local LLM answers, a text-to-speech model reads the answer out. Every piece is swappable, every piece is debuggable, and you already know how to run the middle one. The cost is latency: three hand-offs, typically one to three seconds from the end of your sentence to the first word back on a modest machine, as our offline voice assistant guide shows.
2. A speech-in LLM. Some LLMs in our library take audio directly — Gemma 4 and Nemotron 3 Nano Omni among them. They hear tone and context that a transcript loses, but they still answer in text, so you still need a text-to-speech model for the voice.
3. Full-duplex speech-to-speech. One model listens and speaks at the same time, so it can be interrupted, can back-channel ("mm-hm") and doesn't wait for silence to start thinking. This is how the closed realtime voice products work. The open versions arrived this year, and they're the most interesting part of this guide — and the least practical on consumer hardware.
My rule of thumb from workshops: start with the chain. You'll learn where your latency actually goes, and you can replace one piece at a time.
Speech-to-text: which model listens best
The Open ASR Leaderboard is the reference here: it measures word error rate (WER, lower is better) and speed on the same datasets for every model. Its English average now covers 11 sets, two of them private, so numbers from articles that quote a 7-set average aren't comparable with today's. The rows below are from the board's version of 21 October 2026.
| Model | Size | Languages | English WER (11 sets) | Licence on the weights | Best for |
|---|---|---|---|---|---|
| Qwen3-ASR 1.7B | 1.7B | 52 | 5.40% | Apache-2.0 | Accuracy plus language breadth |
| Granite Speech 5.0 2B | 2B | 6 | 5.68% | Apache-2.0 | The newest strong option (20 October) |
| Cohere Transcribe | 2B | 14 | 5.77% | Apache-2.0 | Commercial English and EU languages |
| Canary-Qwen 2.5B | 2.5B | English | 5.94% | CC-BY-4.0 | English accuracy |
| Parakeet TDT 0.6B v3 | 0.6B | 25 European | 6.18% | CC-BY-4.0 | Raw speed and streaming |
| Qwen3-ASR 0.6B | 0.6B | 52 | 6.24% | Apache-2.0 | Smaller machines |
| Whisper large-v3 | 1.55B | 90+ | 7.16% | MIT | The ecosystem: every runtime supports it |
A few things the table doesn't say:
- Hugging Face itself uses Qwen3-ASR as its reference transcriber — it's the model the new Open TTS Leaderboard uses to check whether TTS output is intelligible, and Hugging Face calls it the top open model on the ASR board. It still is.
- Cohere Transcribe took first place in English when it launched in March and has a clean licence story: Apache-2.0, 2B parameters, a vLLM serving path. Cohere followed it with an Arabic variant in July.
- Parakeet is about speed, not the last decimal of accuracy. On the same board it processes English audio about 6,100 times faster than real time, against roughly 800 for Qwen3-ASR 1.7B. It's also the recognizer in Hugging Face's own speech-to-speech pipeline.
- Whisper large-v3-turbo is no longer the most accurate (7.64%), and it doesn't matter as much as you'd think. It has the most mature local runtimes — whisper.cpp, faster-whisper, WhisperKit on Apple devices — and most local dictation apps are still built on it.
Speech-to-text without a GPU
- Moonshine v2 is built for edge devices: its compute grows with the length of your audio instead of padding everything to a 30-second window like Whisper, and it streams. Synaptics runs it on a small NPU board. The English checkpoints are MIT.
- Fun-ASR-Nano ships as a single llama.cpp-style binary with no Python, in 31 languages. If you transcribe Chinese, FunASR reports roughly 8% character error rate against 22–31% for whisper.cpp on its Mandarin test set — a vendor figure, but a big gap. The GGUF files total under 1 GB.
- Whisper tiny/base through whisper.cpp remain the simplest English option for commands on a Raspberry Pi.
Text-to-speech: quality caught up, licences split three ways
Two leaderboards matter, and they measure different things. The Artificial Analysis Speech Arena is blind human voting: listeners pick the better of two clips. The Open TTS Leaderboard, launched by Hugging Face on 30 September, uses objective metrics — intelligibility (WER of the output), speaker similarity for cloning, and speed — and covers far more open models. Arenas under-sample open weights: only 16 of the 92 models on the Artificial Analysis board were open-weight at the end of September.
The bars below are the Speech Arena Elo of the open models in our catalogue, coloured by what the licence on the weights allows.
On the Speech Arena's main board (read on 25 October 2026), the open-weight models in our catalogue rank like this:
| Model | Maker | Elo | Licence on the weights | Commercial use? |
|---|---|---|---|---|
| Fish Audio Drama 3 Preview | Fish Audio | 1,248 | Fish Audio Community License | Only below a revenue threshold; never as a hosted service |
| Breeze TTS 2 | BreezeBlue | 1,230 | Research and non-commercial | No |
| Ming-3.0-tts 0.5B | inclusionAI | 1,191 | MIT | Yes |
| Fish Audio S2 Pro | Fish Audio | 1,110 | Fish Audio Research License | No |
| Step Audio EditX | StepFun | 1,100 | Not stated (code is Apache-2.0) | Read the model card |
| Voxtral TTS | Mistral AI | 1,091 | CC BY-NC 4.0 | No |
| Kokoro 82M | hexgrad | 1,066 | Apache-2.0 | Yes |
| Magpie-TTS Multilingual 357M | NVIDIA | 1,065 | NVIDIA Open Model License | Yes |
| Higgs Audio V3 TTS | Boson AI | 1,045 | Research and non-commercial | No |
| Chatterbox | Resemble AI | 1,025 | MIT | Yes |
For reference, ElevenLabs' Eleven v3 sits at 1,184 on the same board — below all three of the top open models.
The headline is the licence column, not the Elo column. The best-rated open voice beats a closed flagship and you can ship it only if your company is small enough and you don't host it as a service. The second can't go in a product at all. The third, a 0.5B model from Ant Group's inclusionAI, is MIT. The pattern to watch for: code on GitHub under Apache-2.0, weights under a separate licence. Step Audio EditX is the clearest case — its card licenses "the code in this open-source repository" and says nothing about the weights. The badge on the repository tells you about the code. The model card tells you about the weights, and the weights are what bind you.
What I'd actually pick:
- Kokoro 82M for "read this text in a good voice, cheaply". Apache-2.0, CPU-only if you want, preset voices in several languages, and one of the most-downloaded open TTS models on Hugging Face. No cloning, no emotion control — that's the trade.
- Qwen3-TTS (0.6B or 1.7B, Apache-2.0) for cloning or designing a voice across 10 languages. It's the default TTS in Hugging Face's own speech-to-speech pipeline, and llama.cpp now runs it.
- VoxCPM2 (2B, Apache-2.0) for 30 languages at 48 kHz, in about 8 GB of VRAM per its card.
- Chatterbox (MIT) for cloning from a short reference clip with an emotion dial. Every output carries an inaudible watermark, which is a feature or a constraint depending on your use.
- Ming-3.0-tts (0.5B, MIT) for cloning in seven languages with the best Arena score of the fully commercial models. Its inference code lives in the GitHub repository, not in a standard runtime yet.
- Pocket TTS (Kyutai, 100M, CC-BY-4.0 weights) for voice cloning on a CPU: one reviewer measured about 200 ms on two cores.
- Voxtral TTS if you're cloning a specific voice for a non-commercial project. It needs a 16 GB GPU, and it's from a European lab.
Speech-to-speech: the full-duplex models
Two open models are worth your time right now, and one is worth watching.
NemotronLabs VoiceChat 11B (NVIDIA, 3 August) is the first open full-duplex model that can call tools while the conversation keeps flowing: it emits tool calls on a separate channel and keeps talking while they run. NVIDIA reports 448 ms smooth turn-taking latency on Full-Duplex-Bench 1.0, and Artificial Analysis scored it 38.8% on Big Bench Audio — against 4.3% for Kyutai's Moshi. It's built on a Nemotron Nano 9B v2 backbone, and its weights are under OpenMDW-1.1, which allows commercial use. The listed hardware is A100, H100, H200, B100, B200 or RTX-6000 class on Linux, so for most readers that means renting a GPU.
Need more VRAM than you own? Spin up a cloud GPU big enough for any model in minutes — pay for the hours you use instead of buying a card.
Affiliate links — we may earn a commission if you sign up, at no extra cost to you.
Marketplace pricing — the cheapest per-hour rates for spot/interruptible GPUs.
Rent GPUs on Vast.ai →On-demand pods with a simple UI — good for a quick one-off inference or fine-tune.
Rent GPUs on RunPod →MiniCPM-o 4.5 (OpenBMB, February) is the full-duplex model you can actually run at home. It's a 9B omni model on a Qwen3-8B core that sees, listens and speaks at the same time, under Apache-2.0. For full-duplex streaming through llama.cpp-omni, OpenBMB asks for an NVIDIA GPU with at least 12 GB or an Apple M4 Max with at least 24 GB, and it ships an official Docker image for running it on a Mac.
Dessa (Block, announced) is the one to watch. Block says it is a true speech-to-speech model trained from scratch on speech — no text, no pretrained models — that is full-duplex and runs locally on a MacBook Pro M4 Max or better. As of 25 October there is only a waitlist: no weights, no licence and no benchmark, so it isn't in our tables yet.
Moshi (Kyutai, 2024) started the open full-duplex wave. It handles turn-taking naturally and reasons poorly. Keep it as a reference point, not a recommendation.
The runtime problem: Ollama took its first step this week
For LLMs, one tool gives you pull, run and an API for thousands of models. For voice, that only started on 24 October, when Ollama 0.40.11 added its first speech model: Gradium's Phonon text-to-speech, on Apple Silicon only, served at /v1/audio/speech, with Windows and Linux "coming soon". There is still no speech-to-text in Ollama. llama.cpp moved further: its server has a /tts endpoint (Qwen3-TTS and Pocket TTS) and a /v1/realtime websocket for streaming transcription. What has standardised is the API shape: most servers speak OpenAI's /v1/audio/transcriptions and /v1/audio/speech. The runtimes worth knowing:
| Runtime | Does | No GPU? | Notes |
|---|---|---|---|
| whisper.cpp / faster-whisper | STT | Yes | The defaults; faster-whisper powers our offline assistant guide |
| WhisperKit | STT | Yes | Apple-native; behind many macOS dictation apps |
| FunASR llama.cpp runtime | STT | Yes | One binary, no Python; best for Chinese |
| llama.cpp server | TTS, STT | Yes | /tts for Qwen3-TTS and Pocket TTS; /v1/realtime transcription |
| Ollama 0.40.11+ | TTS | Yes | Phonon only, Apple Silicon only, /v1/audio/speech |
| llama.cpp-omni | TTS, full duplex | Yes | VoxCPM2 and MiniCPM-o 4.5 from GGUF |
| vLLM-Omni | TTS, omni | No | Core vLLM only takes audio in; TTS lives in vllm-omni |
| SGLang-Omni | TTS | No | The path for Drama 3, Higgs Audio V3 and Fish S2 Pro |
| LocalAI | STT, TTS, LLM | Yes | Broadest coverage behind one OpenAI-compatible server |
| Kokoro-FastAPI | TTS | Yes | Fastest route to a Kokoro endpoint |
| Home Assistant Assist + Wyoming | STT, TTS | Yes | Whisper + Piper wired for smart-home voice |
One housekeeping note: Piper moved. The original rhasspy/piper repository is archived; the live project is OHF-Voice/piper1-gpl. Its voices are licensed one by one, so check the voice you pick.
Recipes by hardware
The memory figures below are what each model's maker or a named source publishes — we haven't measured them, and they aren't run through our LLM fit calculator. Check your machine in the analyzer for the LLM in the middle.
No GPU (laptop CPU, Raspberry Pi 5). Ears: Moonshine v2, whisper.cpp base, or Fun-ASR-Nano for Chinese. Mouth: Kokoro, Piper, or Pocket TTS if you want a cloned voice. Brain: a 2–4B model, or put the LLM on another machine on your network and use the Pi as a microphone-and-speaker satellite.
8 GB GPU or 16 GB Mac. Ears: Whisper large-v3-turbo, Parakeet TDT 0.6B v3 or Qwen3-ASR 0.6B. Mouth: Qwen3-TTS 0.6B, Chatterbox or Kokoro — or Phonon in Ollama on a Mac. Brain: a 4–9B model from the 8 GB list. Budget the memory: the three models share it.
16–24 GB GPU or 32 GB+ Mac. Ears: Qwen3-ASR 1.7B, Granite Speech 5.0 or Cohere Transcribe. Mouth: VoxCPM2, Step Audio EditX (12 GB minimum, licence unstated) or Voxtral TTS (16 GB, non-commercial). Or skip the chain and try MiniCPM-o 4.5 full duplex.
This page contains affiliate links.
Server class (rent it). NemotronLabs VoiceChat 11B for full duplex with tools, Drama 3, Fish S2 Pro and Higgs Audio V3 for the top-rated restricted-licence voices. A few hours on a rented data-centre GPU is the cheap way to find out whether full duplex changes anything for your use case.
Voice plus decision models: the trick for smart-home commands
A spoken command like "turn off the kitchen lights" doesn't need a chat model to answer — it needs a fast, reliable choice from a fixed list of devices and actions. That's exactly what decision models do: they return a probability for each allowed option instead of generating text. Community projects already wire them into Home Assistant's voice pipeline, mapping the transcript onto built-in intents and falling back to a chat model only for free-form requests. Since early October, llama.cpp's own server answers these requests locally on a /v1/systemone endpoint, so the whole loop — Whisper, a small decision model, Piper — runs offline on modest hardware. Build a request in the decision request builder.
Honest limits
- We didn't run these models. Every figure comes from a leaderboard, a model card or a named article, with the date it was read. Voice quality is personal: listen before you choose. The Open TTS Leaderboard's "Listen" tab lets you compare outputs side by side.
- Leaderboards disagree by design. Arena Elo measures preference for each maker's chosen voices; the Open TTS Leaderboard measures intelligibility and cloning. A model can lead one and sit mid-table on another.
- Language claims are generous. Test your language.
- Memory numbers here are published, not measured, and three models in a chain share one GPU.
Frequently asked questions
Can I run a voice assistant completely offline?
Yes. Speech-to-text, the LLM and text-to-speech all run locally; you need the internet once, to download the models. Our offline voice assistant guide wires Whisper, an Ollama model and Piper together in about an hour.
What is the best open speech-to-text model in 2026?
On the English Open ASR Leaderboard (version of 21 October 2026), Qwen3-ASR 1.7B has the lowest average word error rate among open models, at 5.40%, ahead of Granite Speech 5.0 2B and Cohere Transcribe. Whisper large-v3 is less accurate but has the broadest runtime support.
What is the best open text-to-speech model I can use commercially?
Among models whose weights allow commercial use without conditions, Ming-3.0-tts (MIT) ranked highest on the Artificial Analysis Speech Arena on 25 October 2026. Kokoro (Apache-2.0) is the best value for fixed voices, and Qwen3-TTS and VoxCPM2 (both Apache-2.0) are the strongest for cloning and many languages.
Can I clone a voice locally?
Yes — Qwen3-TTS, VoxCPM2, Chatterbox, Ming-3.0-tts and Pocket TTS all clone from a short reference clip. Clone only voices you have permission to use; several model licences explicitly prohibit impersonation.
Do I need a GPU for local text-to-speech?
No. Kokoro, Piper and Pocket TTS run on a laptop CPU, and Phonon runs in Ollama on Apple Silicon. A GPU helps for the larger cloning models.
Does Ollama support speech-to-text or text-to-speech?
Text-to-speech, since Ollama 0.40.11 on 24 October 2026: Gradium's Phonon, on Apple Silicon only, at /v1/audio/speech. There is no speech-to-text in Ollama; use whisper.cpp, faster-whisper or llama.cpp's server instead.
What is a full-duplex voice model?
A model that listens and speaks at the same time, so you can interrupt it mid-sentence, rather than waiting for you to stop talking before it starts. NemotronLabs VoiceChat 11B and MiniCPM-o 4.5 are the open examples; Block's Dessa is announced but not downloadable.
Which voice models are open but not free to use commercially?
Breeze TTS 2, Fish Audio S2 Pro, Higgs Audio V3 TTS, Voxtral TTS and OmniVoice have non-commercial weights, and Fish Audio Drama 3 allows commercial use only below a revenue threshold. Their code is often Apache-2.0 — always read the model card, not the repository badge.
Methodology and sources
Researched 25 October 2026. We read leaderboards, model cards and vendor posts; we did not run the models. Leaderboard values carry the date they were read. Where sources disagreed — Parakeet's licence, Pocket TTS's weights licence, Drama 3's licence name — we printed what the model card or its licence file says.
- Open ASR Leaderboard, version 21-10-2026 (English, 11 sets) — Hugging Face, checked 2026-10-25
- Artificial Analysis Speech Arena leaderboard (live table) — Artificial Analysis, checked 2026-10-25
- Open TTS Leaderboard: Scalable Evaluation for Multilingual TTS and Voice Cloning — Hugging Face, checked 2026-10-25
- Speech models in Ollama (Phonon) — Ollama, checked 2026-10-25
- llama.cpp server README (/tts, /v1/realtime, /v1/systemone) — ggml-org, checked 2026-10-25
- llama.cpp TTS README (Qwen3-TTS, Pocket TTS) — ggml-org, checked 2026-10-25
- speech-to-speech README (default pipeline) — Hugging Face, checked 2026-10-25
- rhasspy/piper README ("Development has moved") — GitHub, checked 2026-10-25
- Dessa — Block, checked 2026-10-25
- CohereLabs/cohere-transcribe-03-2026 model card — Hugging Face / Cohere, checked 2026-10-25
- Introducing Cohere-transcribe — Cohere (HF blog), checked 2026-10-25
- Meet Cohere Transcribe Arabic — Cohere (HF blog), checked 2026-10-25
- Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation — arXiv 2510.06961, checked 2026-10-25
- Moonshine v2: Ergodic Streaming Encoder ASR for Latency-Critical Speech — arXiv 2602.12241 (Moonshine AI), checked 2026-10-25
- Real-time Speech Recognition on SL2610 (Moonshine V2 on an NPU) — Synaptics, checked 2026-10-25
- FunASR on PyPI (v1.3.16 changelog, model table) — FunAudioLLM, checked 2026-10-25
- Fun-ASR-Nano GGUF (FunASR llama.cpp runtime) — Hugging Face, checked 2026-10-25
- nvidia/NVIDIA-NemotronLabs-VoiceChat-11B model card — Hugging Face / NVIDIA, checked 2026-10-25
- NVIDIA Releases NemotronLabs VoiceChat, an Open Full-Duplex Voice Model — NYU Shanghai RITS, checked 2026-10-25
- NVIDIA Opens NemotronLabs VoiceChat 11B With Tool Calling — AI Weekly, checked 2026-10-25
- openbmb/MiniCPM-o-4_5 model card — Hugging Face / OpenBMB, checked 2026-10-25
- llama.cpp-omni README (memory table) — GitHub tc-mb, checked 2026-10-25
- Pocket TTS review: Kyutai's 100M CPU voice cloner — DEV Community, checked 2026-10-25
- Qwen/Qwen3-ASR-1.7B model card — Hugging Face, checked 2026-10-25
- ibm-granite/granite-speech-5.0-2b model card — Hugging Face, checked 2026-10-25
- nvidia/canary-qwen-2.5b model card — Hugging Face, checked 2026-10-25
- nvidia/parakeet-tdt-0.6b-v3 model card — Hugging Face, checked 2026-10-25
- fishaudio/drama-3 model card — Hugging Face, checked 2026-10-25
- BreezeBlue/Breeze-TTS-2 model card — Hugging Face, checked 2026-10-25
- inclusionAI/Ming-3.0-tts-0.5B model card — Hugging Face, checked 2026-10-25
- stepfun-ai/Step-Audio-EditX model card — Hugging Face, checked 2026-10-25
- hexgrad/Kokoro-82M model card — Hugging Face, checked 2026-10-25
- Qwen/Qwen3-TTS-12Hz-1.7B-Base model card — Hugging Face, checked 2026-10-25
- openbmb/VoxCPM2 model card — Hugging Face, checked 2026-10-25
- ResembleAI/chatterbox model card — Hugging Face, checked 2026-10-25
- kyutai/pocket-tts model card — Hugging Face, checked 2026-10-25
- mistralai/Voxtral-4B-TTS-2603 model card — Hugging Face, checked 2026-10-25
- openbmb/MiniCPM-o-4_5 model card — Hugging Face, checked 2026-10-25
- nvidia/NVIDIA-NemotronLabs-VoiceChat-11B model card — Hugging Face, checked 2026-10-25
- gradium/phonon-31m-en model card — Hugging Face, checked 2026-10-25