Written by Jakub Rusinowski · Last updated July 10, 2026
A fully offline voice assistant needs three local pieces: Whisper for speech-to-text, a local LLM in Ollama as the brain, and Piper for text-to-speech. All three run on one modest machine, nothing you
A fully offline voice assistant needs three local pieces: Whisper for speech-to-text, a local LLM in Ollama as the brain, and Piper for text-to-speech. All three run on one modest machine, nothing you say leaves the room, and this guide wires them into a working push-to-talk assistant in under an hour.
Last Updated: July 2026
Your voice → [Whisper STT] → text → [Local LLM] → reply text → [Piper TTS] → speakerEach stage is a mature open-source project; the work is plumbing, not research. Set expectations honestly: on an 8GB GPU or an M-series Mac you'll get roughly 1.5–3 seconds from end-of-speech to start-of-reply. That's slower than Alexa's canned responses but *smarter* — and it works with the internet unplugged, which no commercial assistant can claim.
Hardware baseline: any machine that runs a 7–8B model comfortably (8GB VRAM, or a 16GB Apple Silicon Mac) handles the whole stack — check yours in the hardware analyzer.
faster-whisper is the practical Whisper implementation — same accuracy as OpenAI's original, several times faster:
pip install faster-whisper sounddevice numpy requestsModel choice is a speed/accuracy dial:
| Whisper model | Size | Transcription speed | Use when |
|---|---|---|---|
| tiny.en | 39 MB | Instant, even on CPU | Simple commands |
| base.en | 74 MB | Near-instant | The right default |
| small.en | 244 MB | ~1s per utterance (GPU) | Noisy rooms, accents |
| large-v3 | 1.5 GB | Needs a real GPU | Dictation-grade accuracy |
base.en transcribes a 5-second utterance in well under a second on anything modern. (If you only speak commands, tiny.en is genuinely enough.)
Voice interaction is latency-sensitive — a snappy 3–8B model beats a smarter model that makes you wait:
ollama pull llama3.2:3b # fastest good replies
ollama pull llama3.1:8b # smarter, still quick on a GPUGive it a voice-appropriate system prompt (spoken answers must be short — see System Prompts 101):
FROM llama3.2:3b
SYSTEM You are a voice assistant. Answer in 1-3 short conversational sentences. Never use markdown, lists, or code blocks - your output is spoken aloud.ollama create voice -f ModelfileThat "no markdown" line matters: nothing ruins a spoken reply like the TTS engine reading "asterisk asterisk" aloud.
Piper generates natural speech faster than real-time on a Raspberry Pi, let alone your PC:
pip install piper-tts
# Download a voice (one .onnx + one .json file)
python -m piper.download_voices en_US-lessac-medium
# Test it
echo "The assistant is alive." | piper -m en_US-lessac-medium -f test.wavDozens of voices and languages are available; -medium variants are the quality/speed sweet spot. Alternatives: Coqui XTTS sounds more natural but is 10x slower; macOS's built-in say command is a zero-install fallback.
Save as assistant.py — press Enter, speak, get a spoken answer:
import sounddevice as sd
import numpy as np
import requests, subprocess, tempfile, wave
from faster_whisper import WhisperModel
stt = WhisperModel("base.en", compute_type="int8")
SAMPLE_RATE = 16000
def record(seconds=5):
print("Listening...")
audio = sd.rec(int(seconds * SAMPLE_RATE), samplerate=SAMPLE_RATE,
channels=1, dtype="int16")
sd.wait()
return audio
def transcribe(audio):
with tempfile.NamedTemporaryFile(suffix=".wav", delete=False) as f:
with wave.open(f.name, "wb") as w:
w.setnchannels(1); w.setsampwidth(2); w.setframerate(SAMPLE_RATE)
w.writeframes(audio.tobytes())
segments, _ = stt.transcribe(f.name)
return " ".join(s.text for s in segments).strip()
def ask_llm(prompt):
r = requests.post("http://localhost:11434/api/chat", json={
"model": "voice", "stream": False,
"messages": [{"role": "user", "content": prompt}],
})
return r.json()["message"]["content"]
def speak(text):
subprocess.run(["piper", "-m", "en_US-lessac-medium", "-f", "reply.wav"],
input=text.encode())
subprocess.run(["aplay", "reply.wav"]) # macOS: use "afplay"
while True:
input("Press Enter to talk...")
heard = transcribe(record())
if not heard:
continue
print(f"You: {heard}")
reply = ask_llm(heard)
print(f"Assistant: {reply}")
speak(reply)Run python assistant.py, press Enter, and ask it something. Every byte — audio, transcript, reply — stayed on your machine.
silero-vad, one pip install)."stream": true on the Ollama call and pipe sentences to Piper as they complete — the assistant starts speaking while still thinking.The STT and TTS parts, yes — comfortably. The LLM is the bottleneck: a Pi 5 manages ~2–4 t/s on a 3B model, which feels sluggish spoken aloud. The standard pattern is Pi as the microphone/speaker satellite, with Ollama on a real machine over your LAN (see Running on Laptops & Raspberry Pi).
Home Assistant's Assist stack uses the same components under the hood (Whisper + Piper via the Wyoming protocol) aimed at smart-home commands. This guide's loop is the minimal general-purpose version you fully control — and it's the better starting point for understanding what HA automates for you.
Roughly: Whisper base.en 74 MB, Piper voice ~60 MB, Llama 3.2 3B ~2 GB / Llama 3.1 8B ~4.9 GB on disk. At runtime an 8B setup wants ~6 GB of VRAM/unified memory with Whisper sharing happily alongside.
Yes — use a multilingual Whisper model (drop the .en suffix), a Piper voice for your language, and a multilingual LLM (Qwen 2.5 and Gemma 3 are strong picks). All three stages support dozens of languages offline.