How to Set Up a Private AI Assistant for Your Business in 2026

Here's an uncomfortable truth: your employees are probably already using ChatGPT, Claude, or Gemini to handle work tasks. Customer emails, contract drafts, internal documents, financial summaries — all of it is being pasted into third-party AI services. For many businesses, that's a data breach waiting to happen.

The good news: you can deploy a private, self-hosted AI assistant that gives your team the same capabilities as ChatGPT — running entirely on hardware you control, with zero data leaving your network.

What You're Building

A private AI assistant stack with:

• Ollama — runs the AI model on your hardware
• Open WebUI — a ChatGPT-style interface your team accesses via browser
• Persistent chat history — stored on your server, not a third-party cloud
• Document upload and RAG — employees can chat with company documents privately
• User accounts — admin controls who has access

All accessible from any device on your network (or via VPN), requiring no software installation on employee machines.

Hardware Requirements

The hardware depends on your team size and model choice:

For most small businesses (under 20 employees), a single RTX 4090 running Qwen 3 14B handles concurrent requests from 5–10 simultaneous users without queuing issues.

Minimum viable setup: Any modern PC or server with 16 GB RAM, an RTX 4060 (8 GB VRAM), and Ubuntu Server 22.04. Total hardware cost: ~$500–800.

Step 1: Choose and Prepare Your Server

You have two options:

Option A — Repurpose existing hardware. A gaming PC with an RTX 4070 or better works perfectly. Run Ubuntu Server (headless) or Windows Server alongside your GPU drivers. Option B — Buy a dedicated mini-server. Systems like the ASUS NUC Pro or Beelink SER mini PCs with eGPU enclosures work well. Budget $600–1,200 for a complete turnkey setup.

Install Ubuntu Server 22.04 LTS (recommended for best GPU driver support):

# After OS install, update and install GPU drivers
sudo apt update && sudo apt upgrade -y
sudo ubuntu-drivers install

Step 2: Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

# Pull your chosen model
ollama pull qwen3:14b

# Configure Ollama to listen on all interfaces (for network access)
sudo systemctl edit ollama

Add to the systemd override:

[Service]
Environment="OLLAMA_HOST=0.0.0.0:11434"
Environment="OLLAMA_NUM_PARALLEL=4"
Environment="OLLAMA_KEEP_ALIVE=30m"

sudo systemctl daemon-reload && sudo systemctl restart ollama

Step 3: Deploy Open WebUI with Docker

# Install Docker
curl -fsSL https://get.docker.com | sh

# Deploy Open WebUI
docker run -d \
  -p 3000:8080 \
  -e OLLAMA_BASE_URL=http://localhost:11434 \
  -v open-webui:/app/backend/data \
  --name open-webui \
  --restart always \
  ghcr.io/open-webui/open-webui:main

Visit http://your-server-ip:3000 from any browser on the network. The first account you create becomes the admin.

Step 4: Enable RAG for Document Chat

In Open WebUI settings, enable document upload:

1. Go to Admin → Settings → Documents 2. Set Embedding Model to nomic-embed-text (pull it first: ollama pull nomic-embed-text) 3. Employees can now upload PDFs, Word docs, spreadsheets, and chat with them privately

Use cases this unlocks:

• "Summarize this 80-page contract and flag any unusual clauses"
• "Answer questions based on our internal policy documents"
• "Extract the key metrics from these 12 financial reports"
• "Draft a response to this customer email based on our support guidelines"

Step 5: Set Up User Accounts and Permissions

Open WebUI has a built-in user management system:

1. Admin Panel → Users — invite employees by email or create accounts manually 2. Set Default User Role to "User" (not Admin) for security 3. Enable Sign Up Disabled after adding your team — prevents unauthorized access 4. Create model groups if you want some users on faster/better models

Step 6: Secure Remote Access (VPN or Nginx + SSL)

For employees to access from home or on the road, you have two options:

Option A — WireGuard VPN (Recommended): Free, fast, and secure. Employees install WireGuard on their device and connect to your server's VPN. They then access Open WebUI at the internal IP address. Option B — Nginx Reverse Proxy with SSL:

server {
    listen 443 ssl;
    server_name ai.yourcompany.com;
    ssl_certificate /etc/letsencrypt/live/ai.yourcompany.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/ai.yourcompany.com/privkey.pem;
    location / {
        proxy_pass http://localhost:3000;
        proxy_read_timeout 600s;
    }
}

Run certbot --nginx to automatically provision a free SSL certificate from Let's Encrypt.

Total Cost Comparison

For a 15-person team, self-hosting saves $350–360/month vs ChatGPT Team — covering hardware costs in 2–3 months.

Which Model Should You Deploy?

General business use (chat, emails, summaries): Qwen 3 14B — strong reasoning, fast, multilingual. Legal/medical/financial (needs careful reasoning): DeepSeek R1 32B — chain-of-thought transparency, excellent on nuanced tasks. High-volume light tasks: Llama 3.1 8B — fastest throughput, lowest VRAM, handles 20+ simultaneous users on a single 4090. Coding team: Qwen 2.5 Coder 32B — state-of-the-art code completion and review.

Common Questions

Is the quality as good as ChatGPT? For most business tasks (summarization, drafting, Q&A over documents) — yes, Qwen 3 14B or DeepSeek R1 32B are competitive with GPT-4o on typical business writing and analysis. What about compliance? Self-hosted AI is compatible with GDPR, HIPAA (with proper infrastructure controls), and SOC 2 requirements because data never leaves your infrastructure. Can employees use it from their phones? Yes — the Open WebUI interface is fully responsive. Any browser on any device works. Calculate your cost savings with the Cost Calculator → Find the right model for your hardware with the VRAM Checker →