HEALTHCARE · CLINICAL

Healthcare AI: Clinical Documentation and Chart Retrieval on Your Own Hardware

Clinicians are spending their evenings on documentation. The fix cannot be a tool that sends the chart to a stranger.

Written by Jakub Rusinowski · Last updated 2026-08-01 · Hardware figures computed by our VRAM engine

This page is practitioner guidance on deployment mechanics, not legal advice — involve your counsel or DPO for decisions about your specific obligations.

Healthcare AI deployment means putting the model where the clinical data already is, so documentation drafting, chart summarisation, and retrieval over internal protocols happen without disclosing records to a third party. The workloads that pay for themselves are unglamorous — note drafting, discharge summaries, prior-authorisation letters, coding support, and answering questions against your own guidelines — and all of them run comfortably on a single GPU. The hard parts are clinical validation and scope discipline, not model capability.

Why this is hard

What the deployment gives you

Ambient and assisted documentation

Draft notes, discharge summaries, and referral letters from the encounter record, in your templates and your house style. The clinician edits and signs; the model never writes to the chart unattended.

Retrieval over your own protocols

Question-answering grounded in your formulary, care pathways, and policies, with a citation to the source document on every answer. An ungrounded clinical answer is worse than no answer, so grounding is a requirement rather than a feature.

Coding and administrative support

Suggested codes with the supporting passage quoted from the note, prior-authorisation drafting, and denial-letter analysis. Suggestion-with-evidence is the pattern that clears review; autonomous coding is not.

Validated before it goes near a clinic

A held-out set from your own records, reviewed by your own clinicians, with error modes documented. Vendor benchmark scores describe a different population than yours, and the difference is exactly what matters.

Where the model sits in a clinical workflow

The design principle is that the model drafts and retrieves but never decides or commits. Everything it produces lands in front of a clinician as an editable draft with its sources attached, and the write back to the record is a human action.

Where the model sits in a clinical workflowCovered entity — clinical network. 1. EHR / clinical system: System of record. The model reads through an integration; it never writes directly. 2. Retrieval layer: Chart context plus your protocol corpus, scoped to the treatment relationship. 3. Inference server: Open-weight model on your GPUs, inside the clinical network, no egress. 4. Clinician review: Draft with citations. Edit, accept, or discard — the signature stays human.COVERED ENTITY — CLINICAL NETWORKEHR / clinical systemSystem of record. The modelreads through anintegration; it neverwrites directly.Retrieval layerChart context plus yourprotocol corpus, scoped tothe treatment relationship.Inference serverOpen-weight model on yourGPUs, inside the clinicalnetwork, no egress.Clinician reviewDraft with citations. Edit,accept, or discard — thesignature stays human.

What you have to satisfy

Models that fit this deployment

模型VRAM (Q4)可运行于上下文许可
Mistral Small 3.1 24B
Single-clinic tier — 16 GB card — Documentation drafting and retrieval for one site, on hardware that fits an existing server room.
ollama pull mistral-small3.1		15 GB24 GB GPU (RTX 3090/4090)
Mac: 24 GB 统一内存		125KApache 2.0
Qwen 3 32B
Department tier — 24 GB card — The practical default: summarisation, drafting, and grounded Q&A over internal protocols.
ollama pull qwen3:32b		20.6 GB24 GB GPU (RTX 3090/4090)
Mac: 32 GB 统一内存		125KApache 2.0
Llama 3.3 70B Instruct
Health-system tier — 48 GB class — Better handling of clinical nuance and long charts, still a single pro card.
ollama pull llama3.3:70b		43.1 GB2×24 GB GPUs or 48 GB card
Mac: 64 GB 统一内存		128KLlama Community

Cutting documentation burden without moving the chart?

We help healthcare organisations pick the workloads that pay off first, size the hardware honestly, design retrieval that keeps answers grounded, and run the validation their clinical governance process expects. Vendor-neutral: no hardware, no platform, and no model to sell you.

Ask about a workshop →

常见问题

What are the highest-value AI use cases in healthcare right now?
The administrative ones, consistently. Note and discharge-summary drafting, referral and prior-authorisation letters, coding support with the supporting text quoted, and grounded question-answering over internal protocols. They share three properties that make them work: a human reviews the output before it counts, errors are visible rather than silent, and the time saved is measurable in the first month. Diagnostic and triage applications are far more interesting and far more regulated — a different programme with a different risk posture.
Does a clinical AI assistant count as a medical device?
It depends on what it does and how transparently it does it. Under the 21st Century Cures Act criteria and FDA's CDS guidance, software that supports a clinician who can independently review the basis for its output generally sits outside device regulation, while software that directs time-critical care or offers recommendations the clinician cannot interrogate generally does not. Drafting, summarisation, and cited retrieval usually stay on the non-device side. This is a scoping decision to make deliberately with your regulatory lead at design time, because it constrains what you build.
How do you stop a clinical model from making things up?
You constrain what it is allowed to answer from. Retrieval-augmented generation over your own approved corpus, with the source passage shown alongside every answer, converts hallucination from an invisible failure into a visible one — the clinician can see there is no citation, or that the citation does not support the claim. Add explicit refusal behaviour for questions outside the corpus, validate on your own records, and keep a reporting path. What you cannot do is prompt a model into reliability and call that a control.
What hardware does a hospital actually need?
Far less than the budget conversation usually assumes. Clinical usage is bursty and concurrency is low relative to headcount, so a single 24 GB GPU running a 32B-class model comfortably serves a department, and a 48 GB card running a 70B-class model serves a health system for drafting and retrieval workloads. Start by counting concurrent users at peak rather than total staff. Every VRAM figure here comes from the same engine as our compatibility checker, so you can verify a configuration before committing to a purchase.
Should we fine-tune on our own clinical notes?
Usually not as a first move. Retrieval over a curated corpus gets you most of the house-style and local-protocol benefit, keeps the data deletable, and avoids the memorisation risk that comes with training on rare cases. Fine-tuning earns its place for narrow structured tasks — a specific extraction format, a consistent template — after retrieval has plateaued. If you do fine-tune, run it on your own hardware, treat it as a use of PHI with the same governance as any other, and keep the base model available as a fallback.

Go deeper