Air-Gapped AI: Local LLMs on Networks With No Route to the Internet
The environment where "we reviewed the vendor's security posture" is not an acceptable answer, because nothing connects outward at all.
An air-gapped AI deployment runs an open-weight model on hardware inside an isolated network that has no route to the internet, with models, dependencies, and updates crossing the boundary as reviewed media rather than over a link. Large language models suit this unusually well: once the weights are on disk, inference is a pure local computation with no licence check, no telemetry, and no callback — so the capability degrades gracefully into isolation in a way most enterprise software does not.
Why this is hard
Most enterprise AI tooling assumes an outbound connection for licensing, telemetry, or model updates — and fails closed without one.
A model that phones home once at startup is not air-gapped, and you often only discover it during accreditation.
Package installs quietly reach the internet; an environment that has never been built offline usually cannot be.
What the deployment gives you
No outbound route, by construction
The inference host sits on the isolated segment with no default gateway to any external network. Weights load from local storage and inference is a self-contained computation — there is no licence server to reach and nothing to block.
A build that assumes offline from the start
Container images, model files, Python wheels, and CUDA components staged into an internal registry and mirror. The rule that saves projects: if it was not built and tested offline, it does not work offline.
A reviewable update path
Model and dependency updates arrive as signed, checksummed media through your existing transfer procedure. Every crossing is an auditable event with a named approver, which is what accreditation actually asks for.
Verified provenance before the gap
Weights are hashed, scanned, and recorded on the connected side, then verified again inside. Prefer safetensors over pickle-based formats — deserialisation of an untrusted checkpoint is code execution, and inside an enclave that is the worst place to find out.
How the model gets in — and nothing gets out
The whole design reduces to one question: what crosses the boundary, in which direction, under whose approval? Everything on the isolated side is ordinary infrastructure. The interesting engineering is entirely in the transfer step and in having built the enclave to run without a network in the first place.
- Verify checksums on both sides of the gap. A hash computed only where the file was downloaded proves nothing about what arrived.
- Pin exact versions of every dependency, including CUDA and driver builds — "latest" is meaningless in an environment that cannot fetch it.
- Prefer safetensors weights; loading a pickle checkpoint executes code, and an enclave is the last place you want to discover that.
- Keep a rebuild runbook that assumes zero connectivity and test it. The failure mode is discovering the gap during an incident, not during setup.
- Log the transfer events, not just the system events. During accreditation the boundary crossings are what gets examined.
What you have to satisfy
The rules that actually constrain the architecture — cited so you can read them yourself.
NIST SP 800-53SC-7 boundary protectionThe control family that isolation is assessed against; an enclave with a documented, monitored transfer procedure is the pattern assessors expect to see.
IEC 62443-3-3Zones and conduitsThe industrial framing of the same idea — inference belongs in a defined zone with an explicit, minimal conduit rather than a general network path.
NIST SP 800-171Controlled unclassified informationWhere CUI is in scope, isolating the processing environment materially reduces the assessment surface for the AI system itself.
Models that fit this deployment
VRAM figures come from the same engine as our compatibility checker, so every row is independently verifiable.
| Model | VRAM (Q4_K_M) | Runs on | Context | License |
|---|---|---|---|---|
| Qwen 3 14B Compact enclave tier — 12 GB card Where the enclave has limited power and cooling; still capable enough for retrieval and drafting. ollama pull qwen3:14b | 9.7 GB | 12 GB GPU (RTX 3060 12GB / 4070) Mac: 16 GB unified | 125K | Apache 2.0 |
| Qwen 3 32B Standard enclave tier — 24 GB card The default choice: one card, permissive licence, no runtime dependency on anything external. ollama pull qwen3:32b | 20.6 GB | 24 GB GPU (RTX 3090/4090) Mac: 32 GB unified | 125K | Apache 2.0 |
| GPT-OSS 120B High-capability enclave — 2×48 GB When the isolated environment has to match what colleagues get from a frontier API outside. ollama pull gpt-oss:120b | 71.3 GB | 2×48 GB GPUs / big unified memory Mac: 96 GB unified | 128K | Apache-2.0 |
Standing up AI inside an isolated environment?
We help teams design and build air-gapped AI enclaves — staging and transfer procedure, offline dependency mirroring, model selection for constrained hardware, and the runbook your accreditation process will ask to see. Vendor-neutral and practitioner-level.
Frequently asked questions
Can an LLM really run with no internet connection at all?
Yes, and more cleanly than most software. Once the weights are on local disk, generating a token is arithmetic on your GPU — there is no licence check, no telemetry, and no model download at runtime. Serving stacks like vLLM and Ollama run entirely offline provided you staged the model and dependencies beforehand. The parts that break in an air gap are the conveniences built around the model: package installs, auto-updates, and any hosted evaluation or observability service.
How do you update a model in an air-gapped environment?
The same way you update anything else across a boundary: fetch and verify on the connected side, hash it, scan it, put it on approved media, transfer it under your existing procedure, verify the hash again inside, then stage it in the internal registry and roll it out. Because model files are large, plan the media and the window realistically — a 70B-class model in Q4 is tens of gigabytes. Most organisations settle on a scheduled cadence rather than tracking upstream releases.
Is air-gapped the same as on-premise?
No. On-premise means the hardware is yours; it usually still has internet access for updates, telemetry, and identity services. Air-gapped means there is no network route to the outside at all, which changes how you build, install, patch, and monitor everything. Air-gapping is meaningfully more work, so it should follow from a real requirement — classification, safety criticality, export control — rather than from a general preference for security.
What actually breaks when you air-gap an AI deployment?
Four things, predictably. Dependency installation, because package managers reach out by default. Observability, if your monitoring stack ships telemetry to a hosted service. Model evaluation, if your harness downloads benchmark datasets at runtime. And identity, if authentication depends on a cloud directory. Each is solvable with an internal mirror or an enclave-side equivalent, but each has to be solved deliberately before accreditation rather than discovered during it.
Do we need an air gap, or is a segmented network enough?
For most regulated commercial work — healthcare, legal, financial — a well-segmented network with no outbound route from the inference host achieves the substance of the guarantee at a fraction of the operational cost. A true air gap earns its overhead where the threat model includes a determined adversary with network access, or where a classification regime or export-control obligation requires physical isolation. Deciding which of those you are in is the first architectural conversation, not the last.