Written by Jakub Rusinowski · Last updated 2026-07-08 · Prices verified 2026-03-01
Chatbots are the use case where honesty most punctures the local sales pitch: hosted open models are so cheap (Llama 3.1 8B at $0.18 per million tokens serves a busy support bot for ~$22/month) that self-hosting rarely wins on cost — our own calculator shows a budget build never breaking even against same-model hosted pricing. Self-host a chatbot for different reasons: customer conversations that must stay in-house, latency control, freedom from rate limits and provider policy, or volumes beyond a few million tokens/day against frontier-priced backends, where the math flips fast.
A self-hosted bot vs an API backend — same model class on both sides, which is the fair fight.
| Dimension | Local | Cloud |
|---|---|---|
| Cost at 4M tok/day (8B-class) | ~$18/mo electricity (+$41/mo amortization) | ~$22/mo hosted Llama 8B — essentially a tie |
| Cost at 4M tok/day (frontier backend) | n/a (8B runs locally) | ~$570/mo (GPT-4o) — this is where local wins |
| Customer data custody | Conversations never leave your infra | Transcripts transit/reside with the provider |
| Rate limits & concurrency | Your hardware is the only limit | Tier limits; spikes need quota planning |
| Uptime responsibility | Yours: monitoring, restarts, failover | Provider SLA (typically excellent) |
| Content policy control | Your system prompt is the only policy | Provider policies apply to your users' traffic |
| Quality for support flows | 8–14B with RAG over your docs: sufficient for most | Frontier available when flows are genuinely hard |
Busy customer-facing bot: ~4M tokens/day, compared against hosted same-class models — 4M tokens/day, RTX 5060 Ti 16GB Budget Build vs Llama 3.1 8B (Together.ai), electricity $0.15/kWh. Adjust every input in the interactive calculator on this page.
| Cloud cost / month | $21.60 (Llama 3.1 8B, $0.18/M input + $0.18/M output) |
| Local cost / month (24-mo TCO) | $59.04 — $18.00 electricity + hardware amortization |
| Hardware up-front | $985 (RTX 5060 Ti 16GB Budget Build) |
| Break-even | Never at this volume — cloud stays cheaper |
Estimates: 70/30 input/output mix, 24-month amortization, no resale value, load-time electricity only. Cloud prices last verified: 2026-03-01. Hardware street price checked: 2026-07-06.
Most chatbot cost comparisons cheat: they price a local Llama 8B against GPT-4o. Price it against hosted Llama 8B — the same weights, served by Together.ai at $0.18 per million tokens or cheaper on DeepInfra — and a busy 4M-token/day support bot costs about $22/month in the cloud. Our $985 budget build spends ~$18/month on electricity to do the same job: the hardware *never* pays for itself on cost alone. We built the calculator below to show that, not hide it.
So why does anyone self-host a customer bot? Four reasons, all real, none of them "it's cheaper than the same model hosted."
A support chatbot's traffic is customer PII by definition — names, order numbers, complaints, occasionally payment details customers paste despite your warnings. Routing that through any third party creates a processor relationship, a DPA, retention questions, and an entry in your privacy policy (the GDPR page walks through it). Self-hosting makes the answer to "who can see customer conversations?" one word long. For healthcare, finance, and EU-consumer-facing businesses, this alone decides it.
If your flows genuinely need frontier-class handling — nuanced troubleshooting, multilingual empathy, complex policy application — the comparison becomes local-something vs GPT-4o at ~$570/month for our preset volume, and suddenly a 24 GB build running Qwen 3 32B with tight RAG breaks even in months. The trick most production bots use: RAG does the heavy lifting. A 14B model retrieving from *your actual help docs* answers support questions more accurately than a frontier model winging it from general knowledge — see the RAG comparison. Quality-per-dollar for support flows is mostly a retrieval problem, and retrieval is local-friendly.
Rate limits bite bots at the worst time — the product-launch spike, the outage that floods support. Self-hosted concurrency is bounded by your hardware, not your API tier, and batching servers like vLLM sustain surprising concurrent load on one card. Policy control matters more subtly: your users' weird inputs are judged by *your* system prompt, not a provider's evolving content policy; nobody else's moderation layer can misfire on a legitimate customer question about, say, medication dosages or self-harm resources in a healthcare bot.
A hosted bot's cost scales linearly with adoption forever. A self-hosted bot's cost is a step function: the same box serves 10× the traffic until it saturates (then you buy one more box — the calculator models machine count honestly). Startups betting on volume growth like owning the flat part of that curve.
The boring reference stack: vLLM or Ollama serving Qwen 3 14B (16 GB card) or 32B (24 GB card), RAG over your help center (the quality multiplier), a thin orchestration layer with escalation-to-human, and a frontier API key wired in as a fallback route for conversations the local model flags as beyond it — the same router logic that wins for agents. Uptime is the tax: monitoring, restarts, and a failover story are yours now. If that sentence made someone on your team wince, the hosted-8B tier at $22/month is a perfectly honorable answer.
For chatbots, tell yourself the truth the calculator tells: against hosted same-class models, self-hosting never wins on dollars. It wins on custody of customer conversations, on rate-limit and policy independence, and — decisively — whenever the alternative backend is frontier-priced at real volume. Start hosted, instrument your flows, move in-house when transcripts, limits, or a $500+/month frontier bill give you a concrete reason. And whichever side serves the answers: RAG over your own docs improves a support bot more than any model upgrade.
Full list on the cloud AI directory. Buying instead? See /build.
Methodology & assumptions. All cost figures are estimates from one shared model (lib/costCompare.ts): cloud costs = tokens/day × published per-1M-token prices at a 70/30 input/output split × 30 days; local costs = hardware amortized over 24 months with no resale value + electricity for load time only at your rate, with machine count scaled when volume exceeds one machine’s throughput. Cloud prices carry per-entry source URLs and verification dates; hardware prices come from the curated /build catalog (street prices with check dates). Real bills vary with usage mix, discounts, and idle power — treat break-even months as directional, not contractual.