The timeline that matters
The Act entered into force on August 1, 2024 and applies in stages (dates from the Regulation's own text):
| Date | What applies |
|---|---|
| February 2, 2025 | Prohibited practices (Art. 5) and AI-literacy duties |
| August 2, 2025 | Obligations for general-purpose AI (GPAI) model providers; governance structures |
| August 2, 2026 | The main event: obligations for high-risk AI systems (Annex III), transparency duties, most remaining provisions |
| August 2, 2027 | High-risk rules for AI embedded in regulated products (Annex I) and pre-existing GPAI models |
Penalties scale to the violation: up to €35M or 7% of global annual turnover for prohibited practices, lower tiers for other breaches. The reason this page exists on an on-prem hub: the August 2026 date is when the compliance questions stop being theoretical for ordinary companies — and the survey data on the stats page shows procurement already reacting.
The two questions that determine your obligations
What self-hosting changes under the Act — and what it doesn't
- Logging you control. Deployers of high-risk systems must keep system logs. On your own gateway, log format, retention, and completeness are your decisions, already flowing into your SIEM — not a vendor export feature with a retention window you don't set.
- A stable object to assess. Your compliance documentation describes a model version you pin and change on your schedule. API models can change under you; your assessment then describes a moving target.
- No third-country data questions inside the AI workflow. The Act interlocks with GDPR in practice; an on-prem data path removes the transfer-analysis layer from the combined review (the GDPR page covers that half).
- One less party in the accountability chain. Vendor role changes, terms updates, and subprocessor churn all generate re-review work under a combined AI-Act/GDPR posture. A self-hosted open model turns that recurring workstream into a version-pinning decision.
The honest summary: the Act is workload regulation, and self-hosting is evidence infrastructure. Companies choosing on-prem "because of the AI Act" are usually choosing it for the same reason they choose it for GDPR — not exemption, but a shorter, self-owned path to demonstrating the things the law asks them to demonstrate.
A sensible preparation sequence
For a company running (or planning) self-hosted AI, the preparation that counsel will not object to:
1. Inventory and classify every AI use case against Annex III — most will land outside it; the ones inside get the full treatment. 2. Stand up the logging now — the gateway-with-audit-log pattern from the deployment guide satisfies the record-keeping instinct of both this Act and GDPR, and it's a day of work. 3. Pin and document model versions — which model, which weights hash, which system prompts, for each use case. Trivial to do on owned infrastructure; this becomes your technical file's backbone if a use case is high-risk. 4. Add the transparency touches — users told they're talking to AI; AI-literacy training logged (already applicable since February 2025). 5. Put the high-risk use cases through counsel before August 2026 — with the inventory, logs, and version documentation above, that conversation is short.
Frequently asked questions
Does running AI on-premise exempt a company from the EU AI Act?
No. The Act (Regulation (EU) 2024/1689) regulates by use case — a high-risk application carries the same deployer obligations regardless of where the model runs. What self-hosting changes is evidence and dependency: you control the logs, model versions, and data paths your documentation describes, instead of relying on a vendor's. Treat any "on-premise = compliant" claim as a red flag. This is practitioner framing, not legal advice.
When does the EU AI Act actually apply?
In stages: prohibited practices and AI-literacy duties since February 2, 2025; general-purpose AI model-provider obligations since August 2, 2025; the main high-risk system obligations from August 2, 2026; and rules for AI in regulated products plus pre-existing GPAI models by August 2, 2027. The August 2026 date is the one driving current enterprise preparation.
Is an internal LLM assistant "high-risk" under the AI Act?
Generally no — Annex III defines high-risk by domain: employment decisions, education scoring, credit/insurance eligibility, biometrics, essential services, law enforcement. A document-summarization or knowledge assistant falls outside those, facing transparency-level duties instead. The same model becomes high-risk the moment it's wired into, say, CV screening — classification follows the use case, so inventory per use case, not per model.
If we self-host Llama or Qwen, do we take on the Act's GPAI provider duties?
The GPAI model-provider obligations (technical documentation, training-data summaries) sit with the model's developer — Meta, Alibaba, Mistral — not with a company that downloads and deploys the weights, and the Act eases certain duties for open-source-released models. You remain the deployer of your system, with deployer duties scaled to its risk class. Substantially modifying a high-risk system or marketing your own AI product can escalate your role — that boundary is a question for counsel.
What are the EU AI Act penalties?
Tiered by violation type, per the Act's text: up to €35 million or 7% of global annual turnover for prohibited practices, with lower maximums for other non-compliance (up to €15M/3%) and for supplying misleading information (up to €7.5M/1%). SMEs face the lower of the fixed or percentage amounts. The practical takeaway isn't the ceiling — it's that the record-keeping that avoids trouble is cheap to build now.
Keep going
- The infrastructure gap behind the regulation →
- Assess your own deployment against the Act →
- How the Act classifies a deployment — Article 5, 6 and Annex III →
- The GDPR half of the combined compliance picture →
- The gateway-and-logging pattern that becomes your evidence →
- The sovereignty context the Act sits inside →
Rolling this out in your organization?
Jakub Rusinowski, the founder of LLM Configurator, runs corporate workshops and lectures on deploying local LLMs — hardware sizing, model selection, compliance-friendly architectures, and hands-on setup for your team. Direct, vendor-neutral, practitioner-level.