This page is practitioner guidance on deployment mechanics, not legal advice — involve your counsel or DPO for decisions about your specific obligations.
Government AI deployment is governed by where the system boundary sits. A hosted commercial model brings a new external service that must be authorized at the right impact level before agency data touches it; an open-weight model running on infrastructure already inside your authorization boundary inherits the controls and the ATO you have. That is why public bodies with sensitive data and slow procurement cycles increasingly self-host: it converts a multi-quarter authorization problem into a change within an existing boundary.
Why this is hard
Adding an external AI service means a new authorization, a new boundary diagram, and a timeline measured in quarters.
Public records and FOIA obligations attach to AI interactions, and most tooling retains nothing you could produce.
Constituent data in a commercial vendor's logs is a headline risk, not merely a compliance one.
What the deployment gives you
No new authorization boundary
Inference on infrastructure already covered by your ATO means the AI capability lands inside a boundary that has been assessed, rather than beside it. The security review becomes a change to a known system instead of an authorization from scratch.
Records obligations satisfied by default
Prompts, retrieved context, and completions captured into your records system with retention set by schedule. When a records or disclosure request arrives, the material exists and is producible rather than expired in a vendor's rolling log.
Mapped to controls your assessors know
Access control, audit and accountability, boundary protection, and configuration management from NIST SP 800-53 apply to the AI system in exactly the language your assessment already speaks. No new control framework to invent.
Human accountability preserved
Assistive by design: the model drafts and retrieves, a named official decides. Determinations affecting rights, benefits, or entitlements keep a person accountable for the outcome, which is what the governance policies actually require.
Inside the boundary you already had authorized
The design goal is to add capability without adding a boundary. Everything sits within the existing accreditation footprint, and the model artefact is the only thing that comes from outside — reviewed and staged once, like any other software dependency entering an authorized system.
Confirm the impact level of every data category the model can reach through retrieval — the boundary is set by the most sensitive reachable record, not the average one.
Capture interactions into the records system from day one; retrofitting retention after a disclosure request is not a recoverable position.
Keep an accountable official named for each deployed use case, with documented intended use and known limitations.
Publish an internal use policy staff can actually follow, and give them a sanctioned tool — unsanctioned use is what the policy is competing with.
Pin and record the model version with each interaction so a decision can be explained years later.
What you have to satisfy
FedRAMP — Cloud service authorization: A commercial hosted model is a cloud service requiring authorization at the appropriate impact level before agency data reaches it; on-prem inference inside an existing boundary avoids adding one.
NIST SP 800-53 — Security and privacy controls: The control catalogue your assessment already uses — AC, AU, SC, and CM families cover an AI system without needing a separate framework.
OMB M-24-10 — Federal AI governance: Establishes agency AI governance, inventories, and minimum practices for rights- and safety-impacting uses. Know which category a use case falls into before you build it.
44 U.S.C. Ch. 31 — Federal Records Act: Records made or received in the course of business must be preserved per schedule — AI interactions that inform agency action are not exempt from that.
Models that fit this deployment
Model
VRAM (Q4)
Runs on
Context
License
Qwen 3 32B Program office tier — 24 GB card — Drafting, summarisation, and grounded retrieval over regulation and policy material. ollama pull qwen3:32b
20 GB
24 GB GPU (RTX 3090/4090) Mac: 32 GB unified
128,000
Apache 2.0
Llama 3.3 70B Instruct Agency tier — 48 GB class — Longer statutory and case documents; single pro card, version pinned for the record. ollama pull llama3.3:70b
43.1 GB
2×24 GB GPUs or 48 GB card Mac: 64 GB unified
128K
Llama Community
GPT-oss 120B Shared-service tier — 2×48 GB — A department-wide capability where the alternative is a commercial API and a new ATO. ollama pull gpt-oss:120b
We help public bodies deploy AI on infrastructure they already operate — boundary and impact-level analysis, records capture design, hardware sizing, and the control mapping your assessors will work from. Vendor-neutral: no infrastructure, no platform, and no model to sell you.
Can government agencies use commercial AI services?
Sometimes, and the constraint is authorization rather than policy preference. A hosted AI service handling federal data generally needs FedRAMP authorization at an impact level appropriate to that data, and several providers now hold authorizations at the lower and moderate levels. What does not transfer is the authorization itself covering your specific use — the agency still assesses and accepts the risk. For sensitive categories, or where procurement timelines make a new authorization impractical, deploying open weights inside an already-authorized boundary is frequently the faster route.
Do FOIA and public records rules apply to AI prompts?
Treat them as though they do. Records law generally attaches to material made or received in the course of official business, and a prompt that informs an agency decision fits that description comfortably. The practical consequence is architectural: capture the interaction into a records system with retention by schedule, rather than letting it live in a tool's temporary log. Agencies that skip this discover the problem when the first request arrives and the material has already aged out. Confirm the specific classification with your records officer.
What is the difference between GovCloud and on-premise for AI?
Both can sit inside an authorized boundary; they differ in who operates the hardware and how quickly you can scale. Authorized government cloud regions give you elasticity without a datacentre, at the cost of an ongoing provider dependency and a boundary that includes their service. On-premise gives you the tightest control and the simplest boundary story, which matters most for the highest impact levels and for genuinely disconnected sites. Because the weights are portable either way, this is a deployment decision you can revisit without rebuilding.
How do we handle AI use cases that affect people's rights or benefits?
Keep a human accountable for the determination and document the system accordingly. Federal AI governance policy singles out rights-impacting and safety-impacting uses for additional practices — impact assessment, testing, monitoring, and public transparency — so the first step is classifying honestly which category a use case falls into. In practice most successful public-sector deployments start well away from that line: drafting correspondence, summarising case files, and answering staff questions against policy documents, where the official still decides.
Is this affordable for a smaller public body?
More often than expected, because the hardware requirement is modest for the workloads that matter. A single 24 GB GPU running a 32B-class model serves a program office for drafting and retrieval, at a capital cost comparable to a few staff laptops and no per-token bill afterwards. The larger cost is usually the integration and governance work rather than the GPU. Start with one well-scoped use case, size it with our compatibility checker, and expand once the value is demonstrated rather than procuring for a hypothetical peak.