How to Run LLMs Completely Offline — The Air-Gap Guide

Most local AI guides assume you have internet access during setup, occasional updates, and stable connectivity. But some situations demand something stronger: a fully air-gapped setup where nothing — not the model download, not the UI, not a telemetry ping — ever reaches the internet.

This guide covers exactly that: how to download everything you need on a connected machine, transfer it to an air-gapped machine, and run a capable local LLM with no network dependency whatsoever.

Why Air-Gap Your AI?

Security clearance environments — government contractors, defense, intelligence work. Any data sent to a cloud AI could violate operational security or clearance requirements. HIPAA/healthcare — patient data processed by AI must stay within controlled infrastructure. Even "anonymized" data sent to third-party APIs creates compliance risk. Legal and legal discovery — attorney-client privileged documents cannot go through third-party AI services without risk. Local, air-gapped analysis is the only safe approach. Journalism and whistleblower work — sources and documents must never touch corporate servers. Investigative journalists working with sensitive sources need absolute isolation. Industrial control systems — production floor networks are often air-gapped by design. Adding AI assistance to these environments requires offline capability. Extreme privacy preference — for anyone who wants absolute certainty that their thoughts and documents stay private, regardless of any terms of service.

What You Need: The Complete List

Download all of these on a connected machine before moving to the air-gapped environment.

Software

Ollama (model runtime)

• Download from: https://ollama.com/download
• Linux: curl -fsSL https://ollama.com/install.sh — run once connected, then copy the binary
• Windows: Download the .exe installer
• macOS: Download the .dmg

Jan (offline-first interface — recommended for air-gap)

• Download from: https://jan.ai (grab the latest .exe, .dmg, or .AppImage)
• Alternatively: LM Studio (proprietary but works offline after install)

Optional: Open WebUI (for multi-user scenarios)

• Docker image: docker pull ghcr.io/open-webui/open-webui:main
• Save to tar: docker save ghcr.io/open-webui/open-webui:main > open-webui.tar

Models

Pull models while connected, then copy the cache directory:

# Pull your chosen models (examples)
ollama pull llama4:scout          # ~22 GB - best all-rounder
ollama pull qwen2.5:14b           # ~9 GB - fast, multilingual
ollama pull deepseek-r1:8b        # ~5 GB - reasoning tasks
ollama pull nomic-embed-text      # ~274 MB - for RAG/document search

Ollama model storage locations:

• Linux/macOS: ~/.ollama/models/
• Windows: C:\Users\\.ollama\models\

Copy this entire models folder to your transfer drive. On a new machine, placing it in the same path makes models immediately available to Ollama without re-downloading.

Transfer Media

Use encrypted USB drives or encrypted external SSDs. Recommended:

• Apricorn Aegis (hardware-encrypted USB): no software needed, PIN-protected
• VeraCrypt container on any USB: software-encrypted, open source
• Minimum capacity: 64 GB for a basic setup, 256 GB+ for multiple large models

Setting Up on the Air-Gapped Machine

Step 1: Install Ollama (no internet)

Linux:

# Transfer the install script and run with OLLAMA_INSTALL_DIR set
# Or download binary directly from GitHub releases — pick the right architecture
chmod +x ollama-linux-amd64
sudo mv ollama-linux-amd64 /usr/local/bin/ollama
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo systemctl enable ollama
sudo systemctl start ollama

Windows/macOS: Run the installer .exe or .dmg from your transfer drive. No internet connection needed for installation.

Step 2: Restore Your Models

# Copy the models folder from your transfer drive
cp -r /mnt/usb/ollama-models ~/.ollama/models

# Verify models are recognized
ollama list
# Should output your pulled models without connecting to internet

Step 3: Install and Configure Jan

Transfer the Jan installer from your USB. Install and launch it. In Jan settings:

1. Set AI Engine to "Local AI" (not cloud) 2. Jan will auto-detect your Ollama models if Ollama is running, or you can point Jan directly to the model files 3. Disable any telemetry or update check options in Settings → Privacy

Jan is designed for exactly this scenario — it works completely offline and has no mandatory connectivity.

Step 4: Verify Air-Gap (Optional but Recommended)

# Block all outbound traffic at the firewall level (Linux example)
sudo iptables -P OUTPUT DROP
sudo iptables -A OUTPUT -d 127.0.0.1 -j ACCEPT
sudo iptables -A OUTPUT -d 192.168.0.0/16 -j ACCEPT  # Allow LAN only if needed

# Then test that your LLM still works
curl http://localhost:11434/api/generate \
  -d '{"model":"llama4:scout","prompt":"Hello, are you working offline?","stream":false}'

If you get a response, your LLM is running fully locally with no internet requirement.

Document Analysis Without Internet

For processing sensitive documents offline, the full RAG stack can run air-gapped:

# Pull the embedding model while connected (transfer with your other models)
ollama pull nomic-embed-text

# Use Jan's built-in file upload (drag and drop)
# Or use a local script with the Ollama API:
curl http://localhost:11434/api/embed \
  -d '{"model":"nomic-embed-text","input":"Your document text here"}'

For production document workflows, LocalGPT (PromtEngineer on GitHub) is a fully offline RAG pipeline that works with Ollama and runs with no internet dependency.

Keeping Models Updated on an Air-Gapped Machine

The challenge: you can't ollama pull on an isolated machine. Your update workflow:

1. On a connected machine, pull new model versions: ollama pull llama4:scout 2. Copy the updated ~/.ollama/models/blobs/ directory to your USB 3. Transfer to the air-gapped machine and merge with existing blobs 4. Run ollama list to confirm the new version appears

Automate this with a script that checks model dates and only copies changed blobs — most models are layered, so only new layers need transferring for minor updates.

What You Can and Can't Do

Works fully offline:

• Chat with any Ollama-supported model
• Document Q&A and RAG (with local embeddings)
• Code completion and review
• Translation between languages
• Summarization and analysis
• Image understanding (with multimodal models like Llama 4 Scout)

Requires internet (no offline workaround):

• Web search (obviously)
• Real-time information — model knowledge is frozen at its training cutoff
• Voice transcription via cloud APIs (local Whisper works offline, though)
• Pulling new models (must use connected-transfer workflow above)

Security Hardening Checklist

• [ ] Full disk encryption enabled (BitLocker on Windows, FileVault on macOS, LUKS on Linux)
• [ ] BIOS password set to prevent boot from external media
• [ ] USB ports restricted if you're done with the initial transfer
• [ ] Model files verified with checksums before transfer (compare sha256 from Ollama's registry)
• [ ] Jan/Ollama telemetry disabled in settings
• [ ] Firewall rules blocking all outbound traffic except localhost

The Practical Reality

A fully air-gapped local LLM setup running Llama 4 Scout or Qwen 2.5 14B gives you a capable, fast AI assistant that processes information at the speed of your GPU — with absolute certainty that nothing leaves your machine. For security professionals, lawyers, healthcare workers, and privacy-focused users, this isn't paranoia. It's responsible practice.

The setup takes an hour. The peace of mind is permanent.

Find the right model for your hardware with the Hardware Analyzer → Compare model capabilities in the Model Library →