Healthcare AI: Clinical Documentation and Chart Retrieval on Your Own Hardware
Clinicians are spending their evenings on documentation. The fix cannot be a tool that sends the chart to a stranger.
Healthcare AI deployment means putting the model where the clinical data already is, so documentation drafting, chart summarisation, and retrieval over internal protocols happen without disclosing records to a third party. The workloads that pay for themselves are unglamorous — note drafting, discharge summaries, prior-authorisation letters, coding support, and answering questions against your own guidelines — and all of them run comfortably on a single GPU. The hard parts are clinical validation and scope discipline, not model capability.
This page is practitioner guidance on deployment mechanics, not legal advice — involve your counsel or DPO for decisions about your specific obligations.
Why this is hard
Documentation burden is a retention problem now, not just an efficiency one — and it falls hardest on the clinicians you can least afford to lose.
Clinical knowledge is trapped in protocol PDFs and shared drives that nobody can search under time pressure.
Every AI pilot stalls at the same gate: nobody will sign off on patient data leaving the organisation.
What the deployment gives you
Ambient and assisted documentation
Draft notes, discharge summaries, and referral letters from the encounter record, in your templates and your house style. The clinician edits and signs; the model never writes to the chart unattended.
Retrieval over your own protocols
Question-answering grounded in your formulary, care pathways, and policies, with a citation to the source document on every answer. An ungrounded clinical answer is worse than no answer, so grounding is a requirement rather than a feature.
Coding and administrative support
Suggested codes with the supporting passage quoted from the note, prior-authorisation drafting, and denial-letter analysis. Suggestion-with-evidence is the pattern that clears review; autonomous coding is not.
Validated before it goes near a clinic
A held-out set from your own records, reviewed by your own clinicians, with error modes documented. Vendor benchmark scores describe a different population than yours, and the difference is exactly what matters.
Where the model sits in a clinical workflow
The design principle is that the model drafts and retrieves but never decides or commits. Everything it produces lands in front of a clinician as an editable draft with its sources attached, and the write back to the record is a human action.
- Ground every clinical answer in a retrieved document and show the citation; an unsourced answer should be visibly marked as such.
- Validate on your own patient mix before go-live, and re-validate when you change the model — a model swap is a clinical change, not an IT change.
- Keep the scope on the non-device side of clinical decision support unless you are genuinely prepared to take on device obligations.
- Log prompts, retrievals, and completions with clinician identity; these records are PHI and inherit your retention schedule.
- Give clinicians an obvious way to report a bad output, and actually route it somewhere — silent failure modes are how pilots become incidents.
What you have to satisfy
The rules that actually constrain the architecture — cited so you can read them yourself.
21st Century Cures Act §3060Clinical decision support exclusionSoftware avoids device regulation only if the clinician can independently review the basis for its recommendation — which is why citation-grounded output is a regulatory design choice, not a nicety.
FDA CDS guidance (2022)Device vs non-device CDSDraws the line around time-critical decisions and opaque recommendations. Drafting and retrieval sit comfortably outside it; autonomous clinical direction does not.
45 CFR §170.315(b)(11)Decision support interventionsONC certification criteria require source-attribute transparency for predictive interventions in certified health IT — know whether your deployment falls inside that scope.
45 CFR §164.312HIPAA technical safeguardsAccess control, audit, integrity, and transmission security apply to the AI system exactly as they do to the EHR beside it.
Models that fit this deployment
VRAM figures come from the same engine as our compatibility checker, so every row is independently verifiable.
| Model | VRAM (Q4_K_M) | Runs on | Context | License |
|---|---|---|---|---|
| Mistral Small 3.1 24B Single-clinic tier — 16 GB card Documentation drafting and retrieval for one site, on hardware that fits an existing server room. ollama pull mistral-small3.1 | 15 GB | 24 GB GPU (RTX 3090/4090) Mac: 24 GB unified | 125K | Apache 2.0 |
| Qwen 3 32B Department tier — 24 GB card The practical default: summarisation, drafting, and grounded Q&A over internal protocols. ollama pull qwen3:32b | 20.6 GB | 24 GB GPU (RTX 3090/4090) Mac: 32 GB unified | 125K | Apache 2.0 |
| Llama 3.3 70B Instruct Health-system tier — 48 GB class Better handling of clinical nuance and long charts, still a single pro card. ollama pull llama3.3:70b | 43.1 GB | 2×24 GB GPUs or 48 GB card Mac: 64 GB unified | 128K | Llama Community |
Cutting documentation burden without moving the chart?
We help healthcare organisations pick the workloads that pay off first, size the hardware honestly, design retrieval that keeps answers grounded, and run the validation their clinical governance process expects. Vendor-neutral: no hardware, no platform, and no model to sell you.
Frequently asked questions
What are the highest-value AI use cases in healthcare right now?
The administrative ones, consistently. Note and discharge-summary drafting, referral and prior-authorisation letters, coding support with the supporting text quoted, and grounded question-answering over internal protocols. They share three properties that make them work: a human reviews the output before it counts, errors are visible rather than silent, and the time saved is measurable in the first month. Diagnostic and triage applications are far more interesting and far more regulated — a different programme with a different risk posture.
Does a clinical AI assistant count as a medical device?
It depends on what it does and how transparently it does it. Under the 21st Century Cures Act criteria and FDA's CDS guidance, software that supports a clinician who can independently review the basis for its output generally sits outside device regulation, while software that directs time-critical care or offers recommendations the clinician cannot interrogate generally does not. Drafting, summarisation, and cited retrieval usually stay on the non-device side. This is a scoping decision to make deliberately with your regulatory lead at design time, because it constrains what you build.
How do you stop a clinical model from making things up?
You constrain what it is allowed to answer from. Retrieval-augmented generation over your own approved corpus, with the source passage shown alongside every answer, converts hallucination from an invisible failure into a visible one — the clinician can see there is no citation, or that the citation does not support the claim. Add explicit refusal behaviour for questions outside the corpus, validate on your own records, and keep a reporting path. What you cannot do is prompt a model into reliability and call that a control.
What hardware does a hospital actually need?
Far less than the budget conversation usually assumes. Clinical usage is bursty and concurrency is low relative to headcount, so a single 24 GB GPU running a 32B-class model comfortably serves a department, and a 48 GB card running a 70B-class model serves a health system for drafting and retrieval workloads. Start by counting concurrent users at peak rather than total staff. Every VRAM figure here comes from the same engine as our compatibility checker, so you can verify a configuration before committing to a purchase.
Should we fine-tune on our own clinical notes?
Usually not as a first move. Retrieval over a curated corpus gets you most of the house-style and local-protocol benefit, keeps the data deletable, and avoids the memorisation risk that comes with training on rare cases. Fine-tuning earns its place for narrow structured tasks — a specific extraction format, a consistent template — after retrieval has plateaued. If you do fine-tune, run it on your own hardware, treat it as a use of PHI with the same governance as any other, and keep the base model available as a fallback.