ISOLATED NETWORKS

Air-Gapped AI: Local LLMs on Networks With No Route to the Internet

The environment where "we reviewed the vendor's security posture" is not an acceptable answer, because nothing connects outward at all.

Written by Jakub Rusinowski · Last updated 2026-08-01 · Hardware figures computed by our VRAM engine

An air-gapped AI deployment runs an open-weight model on hardware inside an isolated network that has no route to the internet, with models, dependencies, and updates crossing the boundary as reviewed media rather than over a link. Large language models suit this unusually well: once the weights are on disk, inference is a pure local computation with no licence check, no telemetry, and no callback — so the capability degrades gracefully into isolation in a way most enterprise software does not.

Why this is hard

What the deployment gives you

No outbound route, by construction

The inference host sits on the isolated segment with no default gateway to any external network. Weights load from local storage and inference is a self-contained computation — there is no licence server to reach and nothing to block.

A build that assumes offline from the start

Container images, model files, Python wheels, and CUDA components staged into an internal registry and mirror. The rule that saves projects: if it was not built and tested offline, it does not work offline.

A reviewable update path

Model and dependency updates arrive as signed, checksummed media through your existing transfer procedure. Every crossing is an auditable event with a named approver, which is what accreditation actually asks for.

Verified provenance before the gap

Weights are hashed, scanned, and recorded on the connected side, then verified again inside. Prefer safetensors over pickle-based formats — deserialisation of an untrusted checkpoint is code execution, and inside an enclave that is the worst place to find out.

How the model gets in — and nothing gets out

The whole design reduces to one question: what crosses the boundary, in which direction, under whose approval? Everything on the isolated side is ordinary infrastructure. The interesting engineering is entirely in the transfer step and in having built the enclave to run without a network in the first place.

How the model gets in — and nothing gets outAir-gapped enclave — no network route in or out. 1. Staging host: Connected side: fetch weights, mirror dependencies, hash and scan everything. 2. Transfer media: Signed, checksummed, one-way. Every crossing logged with an approver. 3. Internal registry: Enclave-side mirror of images, wheels, and model files. The only install source. 4. Inference server: vLLM or Ollama on local GPUs. No gateway, no telemetry, no licence check. 5. Internal clients: Enclave workstations over the internal network, authenticated to your directory.AIR-GAPPED ENCLAVE — NO NETWORK ROUTE IN OR OUTStaging hostConnected side: fetchweights, mirrordependencies, hash and scaneverything.Transfer mediaSigned, checksummed,one-way. Every crossinglogged with an approver.Internal registryEnclave-side mirror ofimages, wheels, and modelfiles. The only installsource.Inference servervLLM or Ollama on localGPUs. No gateway, notelemetry, no licencecheck.Internal clientsEnclave workstations overthe internal network,authenticated to yourdirectory.

What you have to satisfy

Models that fit this deployment

模型VRAM (Q4)可运行于上下文许可
Qwen 3 14B
Compact enclave tier — 12 GB card — Where the enclave has limited power and cooling; still capable enough for retrieval and drafting.
ollama pull qwen3:14b		9.7 GB12 GB GPU (RTX 3060 12GB / 4070)
Mac: 16 GB 统一内存		125KApache 2.0
Qwen 3 32B
Standard enclave tier — 24 GB card — The default choice: one card, permissive licence, no runtime dependency on anything external.
ollama pull qwen3:32b		20.6 GB24 GB GPU (RTX 3090/4090)
Mac: 32 GB 统一内存		125KApache 2.0
GPT-OSS 120B
High-capability enclave — 2×48 GB — When the isolated environment has to match what colleagues get from a frontier API outside.
ollama pull gpt-oss:120b		71.3 GB2×48 GB GPUs / big unified memory
Mac: 96 GB 统一内存		128KApache-2.0

Standing up AI inside an isolated environment?

We help teams design and build air-gapped AI enclaves — staging and transfer procedure, offline dependency mirroring, model selection for constrained hardware, and the runbook your accreditation process will ask to see. Vendor-neutral and practitioner-level.

Ask about a workshop →

常见问题

Can an LLM really run with no internet connection at all?
Yes, and more cleanly than most software. Once the weights are on local disk, generating a token is arithmetic on your GPU — there is no licence check, no telemetry, and no model download at runtime. Serving stacks like vLLM and Ollama run entirely offline provided you staged the model and dependencies beforehand. The parts that break in an air gap are the conveniences built around the model: package installs, auto-updates, and any hosted evaluation or observability service.
How do you update a model in an air-gapped environment?
The same way you update anything else across a boundary: fetch and verify on the connected side, hash it, scan it, put it on approved media, transfer it under your existing procedure, verify the hash again inside, then stage it in the internal registry and roll it out. Because model files are large, plan the media and the window realistically — a 70B-class model in Q4 is tens of gigabytes. Most organisations settle on a scheduled cadence rather than tracking upstream releases.
Is air-gapped the same as on-premise?
No. On-premise means the hardware is yours; it usually still has internet access for updates, telemetry, and identity services. Air-gapped means there is no network route to the outside at all, which changes how you build, install, patch, and monitor everything. Air-gapping is meaningfully more work, so it should follow from a real requirement — classification, safety criticality, export control — rather than from a general preference for security.
What actually breaks when you air-gap an AI deployment?
Four things, predictably. Dependency installation, because package managers reach out by default. Observability, if your monitoring stack ships telemetry to a hosted service. Model evaluation, if your harness downloads benchmark datasets at runtime. And identity, if authentication depends on a cloud directory. Each is solvable with an internal mirror or an enclave-side equivalent, but each has to be solved deliberately before accreditation rather than discovered during it.
Do we need an air gap, or is a segmented network enough?
For most regulated commercial work — healthcare, legal, financial — a well-segmented network with no outbound route from the inference host achieves the substance of the guarantee at a fraction of the operational cost. A true air gap earns its overhead where the threat model includes a determined adversary with network access, or where a classification regime or export-control obligation requires physical isolation. Deciding which of those you are in is the first architectural conversation, not the last.

Go deeper