The gap between the diagram and the box
An accreditation boundary is written in network terms. DISA's Cloud Computing Security Requirements Guide defines the DoD Impact Levels — IL2 for publicly releasable information and other unclassified information not designated CUI, IL4 for CUI on non-critical missions and non-national-security systems, IL5 for higher-sensitivity CUI and national security systems, IL6 for information classified up to SECRET — and the artifact you get from a program office is a boundary diagram with arrows that stop at an edge. That document is correct and it is not the thing that breaks. What breaks is a Python library, three dependencies down, that has been quietly reaching out to the internet on every developer laptop it has ever run on.
The mechanism is uninteresting and extremely reliable. On a connected workstation, the first run of a stack populates a cache. The code path that fetched the weights, the merges file, the sentence-tokenizer data, the CA bundle — that path executed once, months ago, and has not executed since. Every subsequent run reads from disk and looks offline. The path runs again exactly once more: on a clean host, inside the boundary, on the morning of the install, with no way to fetch and no way to bring a laptop into the room.
So the useful question is not "is the system air-gap capable." Every vendor says yes to that. The useful question is: what is the complete list of hosts this software attempts to contact, and what does each attempt do when it fails? That list is discoverable in an afternoon on a bench, and it is the single highest-value artifact you can hand an install team.
Air-gapped, as an engineering constraint
No outbound route to any network the vendor controls; no DNS resolution beyond an internal resolver; no package index, no container registry, no model hub, no license server, no telemetry sink, no certificate revocation responder. Anything the system needs must already be inside the boundary or must arrive through a controlled media-transfer procedure with a human, an approval, and a cadence measured in days.
Where the network call hides
Below is the enumeration we actually walk when packaging an inference stack for a disconnected enclave. Every row is behavior documented by the project that owns it, not something inferred from a stack trace.
| Layer | What it reaches for | The documented offline lever |
|---|---|---|
| Model weights and config | Hub resolve calls on every from_pretrained, including revision checks against a cached copy. |
Hugging Face documents HF_HUB_OFFLINE=1, which makes no HTTP calls to the Hub and reads only cached files; it also skips HTTP verification calls and disables the library's telemetry. Pre-populate the cache directory and mount it read-only. |
| Tokenizer / BPE data | OpenAI's tiktoken fetches encoding files from openaipublic.blob.core.windows.net the first time an encoding is initialized. |
Set TIKTOKEN_CACHE_DIR to a persistent directory pre-seeded with the encoding files, each named for the SHA-1 of its source URL — the naming is the part people get wrong. Left unset, tiktoken caches under the system temp directory, which is empty in a fresh container. |
| Classical NLP assets | NLTK corpora such as the sentence tokenizer are fetched by nltk.download(); spaCy pipelines are fetched by spacy download. |
Ship the nltk_data tree to a path NLTK already searches, and install spaCy pipelines from the .whl or .tar.gz archive published in the spacy-models repository. |
| Telemetry | Product telemetry that is on by default in the version you pinned. Chroma collected anonymized product telemetry until it stopped in version 1.5.4 — a pinned earlier release still sends it. | Read the docs for the version you are shipping, not the current release. Chroma's documented opt-out is anonymized_telemetry=False in Settings, or the ANONYMIZED_TELEMETRY environment variable. Assume every other component in the stack has an equivalent and go find it. |
| Python dependency resolution | Installing a local wheel still consults the index to resolve that wheel's dependencies. | Build a complete wheelhouse and install with pip install --no-index --find-links. A local file path on the command line is not the same thing as an offline install. |
| GPU licensing | NVIDIA vGPU software licensing normally serves licenses from a license service. | NVIDIA documents a Delegated License Service instance hosted on-premises, and node-locked licenses obtained from a file installed locally on a client with no network connection. Both require licenses to be downloaded from the licensing portal and uploaded manually. |
| TLS trust and revocation | Internal services signed by an internal CA; some clients also attempt an OCSP revocation check (RFC 6960). | Install the enclave CA bundle and point the runtime at it explicitly. Confirm what the TLS client does when the revocation responder is unreachable — soft-fail is common, but the attempt still costs a timeout. |
The failure mode is a hang, not an error
This is the detail that decides whether an install goes badly or goes very badly. A blocked outbound call in a well-behaved network gets a refusal and raises an exception immediately. A blocked outbound call in a default-deny enclave that drops packets rather than rejecting them produces nothing at all. The client waits.
How long it waits is a library question, and the most widely used HTTP client in the Python ecosystem answers it badly by default: requests sets no timeout unless the caller passes one, so a request against a silently dropped route waits indefinitely. Almost nobody passes one inside a transitive dependency. The visible symptom is that a service started, logged its banner, and then did nothing — which is a far worse thing to debug in a facility you cannot bring tooling into than a stack trace naming a hostname would have been.
The design rule that follows is small and worth enforcing: in a build intended for a disconnected environment, every outbound call gets an explicit and short timeout, or it does not exist. If a dependency will not let you set one, that is information about the dependency.
Find out on a bench, not on site
The test is cheap. Docker's none network driver creates only a loopback interface inside the container, with no other interfaces and no route to the outside world. Run the image with --network none and see whether the workload completes. Two conditions make the result mean something:
Start from a cold cache. Wipe the model cache, the tokenizer cache, the NLP data directory and the package cache before the run. A warm cache is the single most common reason a stack passes on the bench and fails on site; the machine you are testing on has already done, months ago, all the fetching you are trying to detect.
Exercise the whole lifecycle, not the startup. Many fetches are lazy — they fire the first time a particular code path runs, not at import. Cold start, first request, the long-document path, the batch job, the nightly index rebuild, the restart-after-crash path. A service that boots offline and then reaches for a tokenizer on the first unusual input has not been tested.
There is a more informative intermediate step, and we run it before the fully isolated one. Instead of removing the network, put the container behind an egress policy that logs and drops. Now the run produces the artifact you actually wanted: a list of every host the software tried to reach, in order, with timestamps. That list goes into the install package. It is also the right thing to hand a security reviewer, because it converts "we believe it is offline-capable" into an enumeration somebody else can check.
The hardware assumption that also does not travel
Network egress is the famous problem. The quieter one is that an optimized artifact is often bound to the machine that produced it. NVIDIA's TensorRT documentation is explicit: by default, serialized engines are only guaranteed to work correctly with the same operating system, CPU architecture, GPU model and TensorRT version used to serialize them. Hardware-compatibility mode relaxes the GPU constraint by excluding architecture-specific tactics, at a documented cost in throughput or latency.
The practical consequence is that "we built the engine and it is in the tarball" is not a deliverable unless the target GPU matches the build GPU. Either the build runs on representative hardware, or the build step itself — with its full toolchain, at the same versions — ships inside the boundary and runs there. The second option is more work and it is usually the right one, because it survives the target hardware changing between the design review and the install.
What to establish before agreeing to an install date
Most air-gap schedule overruns we have seen described are not inference problems. They are transfer-procedure problems that were discovered late. These are the questions worth answering in writing first:
- The exact target — GPU model and count, driver version, container runtime and version, OS baseline and patch level. Not the family. The version.
- Drop or reject — whether the boundary silently drops or actively refuses, and whether DNS resolves at all inside it. This determines whether failures are loud or invisible.
- What already lives inside — an internal package mirror, container registry or artifact store, and who is authorized to write to it.
- The media-transfer procedure — how a file gets in, who approves it, and the realistic turnaround. This is usually the true schedule driver, not any engineering task.
- Trust and time — where the enclave CA bundle lives, and what the internal time source is. Clock skew breaks token and certificate validation in ways that look like application bugs.
- Who signs the evidence — which named person accepts the "this ran with zero egress" artifact, and in what format they need it.
Where the line is for us
Stating scope plainly is more useful than claiming breadth, so here is ours on this class of work.
We do not sell an authorization
We build systems designed to be accreditable and we produce the evidence an assessment needs. An authorization to operate is granted by a government authorizing official against a specific system in a specific boundary. We are not that official, we do not hold an ATO on your behalf, and any firm that describes an impact level as a product it sells you is describing something else.
We will not "make it work on site"
If the first time a stack runs disconnected is inside the facility, the install has become a debugging session with the wrong tools and an audience. We run the isolated-network test on a bench first, and if the schedule does not allow for that, we say so before the date is set rather than after it slips.
We will not ship a dependency whose network behavior we cannot enumerate
If a component's outbound calls are not inspectable — obfuscated binaries, undocumented callbacks, telemetry with no documented switch — we replace it or we tell you the capability is not available under that constraint. A shorter list of things that provably do not phone home beats a longer list we cannot vouch for.
We do not do the facility side
Clearance sponsorship, secure-space accreditation, cross-domain solution engineering and media-transfer accreditation are separate disciplines with their own firms. We work to the procedure your organization already has; we do not stand one up.
What survives the air gap
The architecture that comes through this constraint intact is a narrow one, and the constraint is the reason we build the way we do: small models that read through a body of data and produce a written conclusion, with every statement traced back to the exact record it came from. Both halves matter here. A small model fits on hardware an enclave already has and can be shipped as bytes rather than as a service. A citation that points to a record inside the boundary is verifiable by the reader at their desk, with no call to anything.
A system whose output can only be trusted by asking an external service to vouch for it does not survive an air gap, no matter how good the model is. A system whose output points at the source document does — and it is also, not coincidentally, the system a reviewer can audit.
Frequently asked questions
Run it from a cold cache with the network stack removed — Docker's none driver creates only a loopback interface — and exercise the entire lifecycle, not just startup. Before that, run it once behind a log-and-drop egress policy so you get an explicit list of every host it attempted to contact. That list, not a claim, is the deliverable.
Because the laptop has a warm cache. The fetch happened once and has not repeated since, so the code path that requires the network is invisible during normal development. On a clean host inside a boundary, that path runs for the first time in months and there is nothing to fetch from.
Model-hub resolve calls on load, tokenizer and BPE file downloads, NLP corpus downloads, anonymized product telemetry, dependency resolution during package install, GPU license checks, and certificate revocation lookups. Each has a documented offline configuration; the failure is almost always that nobody enumerated the list.
Not by default. NVIDIA documents that serialized TensorRT engines are only guaranteed to work with the same OS, CPU architecture, GPU model and TensorRT version used to build them, unless hardware-compatibility mode is used — which trades some performance for portability. Plan to build on representative hardware or to run the build inside the boundary.
Usually the media-transfer procedure, not the engineering. Every correction after the first transfer costs another approval cycle. That is why the enumeration and the isolated-network test happen before anything is packaged: the goal is to need one transfer instead of several.