This page is practitioner guidance on deployment mechanics, not legal advice — involve your counsel or DPO for decisions about your specific obligations.
Legal AI deployment means running the model inside the firm or legal department so privileged and confidential material is never disclosed to an outside service. Self-hosting removes the waiver argument entirely — there is no third party to have disclosed to — and it makes matter-level access control enforceable in the retrieval layer, where an ethical wall belongs. The two remaining risks are citation fabrication and scope creep, and both are architecture problems with known answers.
Why this is hard
Confidentiality duties attach to the information, not to how convenient the tool is; a public chatbot is an outside recipient.
Courts have sanctioned lawyers for filings containing citations that do not exist — the failure mode is confident and invisible.
Ethical walls are enforced in your document system and then quietly bypassed by a firm-wide AI index nobody scoped.
What the deployment gives you
No third-party disclosure to argue about
Inference runs on hardware the firm administers, so privileged material never reaches an outside provider. The waiver analysis that follows a cloud AI deployment simply does not arise, which is the shortest possible answer to give a client or a court.
Matter scoping as an enforced boundary
Retrieval is restricted to the matters the requesting user is already cleared for, enforced in the index rather than the prompt. This is how an ethical wall survives contact with an AI assistant instead of being undone by it.
Citation-grounded output
Every assertion carries a pointer to the source passage in the corpus it came from. A claim without a retrievable citation is displayed as unsupported, which turns fabrication from a hidden defect into a visible one.
Review and drafting at document scale
Summarise and classify across large productions, extract obligations and dates from contracts, and draft from the firm's own precedent bank — with the throughput coming from your own GPUs rather than a per-token bill.
Where privileged material goes, and where it stops
Everything privileged stays inside the firm boundary. The one component sourced externally is the model file itself, which arrives once and contains none of your client material — so there is no ongoing relationship with an outside party to disclose to.
Verify every citation before it leaves the firm. Grounding reduces fabrication; it does not remove the professional obligation to check.
Partition the index by matter and client from day one — retrofitting ethical walls onto a firm-wide index is far harder than building them in.
Set retention on prompts and completions deliberately; they contain client confidences and belong in your records schedule.
Record which matters an AI system touched, so you can answer a client asking whether their material was processed and how.
Keep the model out of the filing path. Drafting assistance is a supervised act; nothing generated should reach a court without a lawyer's review.
What you have to satisfy
ABA Model Rule 1.6(c) — Confidentiality of information: Requires reasonable efforts to prevent unauthorised disclosure of client information — which makes the destination of an uploaded document an ethics question, not just an IT one.
ABA Model Rule 1.1, cmt. 8 — Technology competence: Competence includes understanding the benefits and risks of relevant technology, so "I did not know it sent the document out" is not a defence.
ABA Model Rule 5.3 — Supervision of nonlawyer assistance: Work delegated to a tool still requires supervision proportionate to the risk — the basis for treating AI output as a draft requiring review.
FRE 502(d) — Privilege and inadvertent disclosure: Clawback orders limit waiver in discovery, but the cleanest position is the one where no disclosure to an outside party ever occurred.
Models that fit this deployment
Model
VRAM (Q4)
Runs on
Context
License
Qwen 3 32B Practice-group tier — 24 GB card — Summarisation, extraction, and grounded Q&A over matter documents on a single card. ollama pull qwen3:32b
20 GB
24 GB GPU (RTX 3090/4090) Mac: 32 GB unified
128,000
Apache 2.0
Llama 3.3 70B Instruct Firm tier — 48 GB class — Better long-document reasoning for contract analysis and review at scale. ollama pull llama3.3:70b
43.1 GB
2×24 GB GPUs or 48 GB card Mac: 64 GB unified
128K
Llama Community
GPT-oss 120B Large-firm tier — 2×48 GB — Where drafting quality has to stand comparison with what associates get from frontier APIs. ollama pull gpt-oss:120b
Putting AI on privileged material without disclosing it?
We help firms and in-house legal teams deploy AI on their own infrastructure — matter-scoped retrieval and ethical-wall design, citation grounding, hardware sizing for review-scale throughput, and the write-up your risk committee will ask for. Vendor-neutral and practitioner-level.
Does using AI on privileged documents waive privilege?
The risk turns on disclosure to a third party. Sending privileged material to an outside AI service is a disclosure, and whether it waives privilege depends on the terms, the jurisdiction, and the precautions taken — an argument you would rather not have to make. Running the model on hardware the firm administers means no outside party received anything, so the question does not arise. That is the practical reason legal departments and firms end up self-hosting even when a vendor offers reassuring contract language.
How do you stop legal AI from inventing case citations?
Constrain the model to a corpus and show its work. Retrieval-augmented generation over an authoritative source set, with the retrieved passage displayed next to each assertion, means an uncited claim is visibly uncited rather than silently wrong. Add refusal behaviour when nothing relevant is retrieved. None of this removes the lawyer's duty to verify every citation before filing — the architecture makes verification fast and the failures obvious, which is the most a system can honestly promise.
Can a self-hosted model handle contract review at real volume?
Yes, and volume is where owning the hardware starts to pay. Review workloads are throughput problems — thousands of documents, similar prompts — which is exactly the profile that makes a metered per-token bill uncomfortable and a fixed GPU cost attractive. A 70B-class model on a single pro card sustains meaningful batch throughput, and you can run it overnight without watching a meter. Extraction accuracy should be validated on your own document types before you rely on it.
What about ethical walls when everything is in one index?
This is the failure mode to design against from the start. If you build a single firm-wide index, an assistant can surface material across a wall that your document management system carefully enforces. Partition the index by matter and client, resolve access from the same directory groups that already govern the document system, and enforce it in the retrieval layer rather than by instructing the model. Retrofitting this later is genuinely difficult, so it belongs in the first architecture conversation.
Do we need to tell clients we use AI on their matters?
Increasingly, yes — and engagement letters are catching up. Disclosure expectations vary by jurisdiction and by client, and many sophisticated clients now impose their own AI terms through outside counsel guidelines. The practical position is to know exactly which systems touch client material, keep a record of which matters were processed, and be able to state plainly that nothing left the firm. Self-hosting makes that a much easier conversation. Confirm the specific obligations with your professional responsibility counsel.