HIPAA isn't scary because it's complicated. It's scary because it's simple, and simple rules plus clever engineers is how you end up on an OCR press release instead of in prod.
I worked at a SaaS company that advertised “AI” for financial automation. Their support team refused to show me how it worked unless I drove to their office and watched it live. So I drove over, sat in a little conference room, and watched someone click through their “AI pipeline.” Later I got access to the code. Surprise: most of the “AI” was column mapping and data formatting wrapped around a spreadsheet-flavored rule engine. It was just spreadsheet glue with a marketing budget.
Healthcare software has the same trap. You can ship something that looks like an AI product and accidentally build a PHI forwarding machine.
Privacy Rule vs Security Rule (dev translation)
HIPAA splits into two buckets that matter for programmers: Privacy and Security. hhs
- Privacy Rule: who can use or disclose protected health information (PHI), and for what reason. hipaajournal
- Security Rule: how you protect electronic PHI (ePHI) with technical, physical, and administrative safeguards. hipaajournal
Programmers usually start with Security because it looks like normal work: encryption, auth, networks, secrets, logs. Privacy is the part that sneaks in through “small” product decisions: what you send to vendors, what you keep, for how long, and who gets to see it. accountablehq
ePHI: yes, your chatbot is in it
If your system touches patient-identifying data tied to health info, you're in ePHI territory. A symptom list plus a name. A phone number plus “needs insulin refill.” “My MRI results looked bad, can you explain this?” in a chat box. metomic
Chatbots and agents are PHI magnets. Users will paste entire discharge summaries, labs, portal messages, all of it, because from their perspective your text box is just “helpful assistant that understands everything.”
If your app routes that text into an LLM API, you're now creating, transmitting, and processing ePHI. The feature label (“assistant,” “copilot,” “triage bot”) does not matter. The data flowing through it does. niu
The vendor landmine
Three quick realities:
- If you're a covered entity or a business associate, HIPAA sticks to you. hhs
- If you send PHI to a vendor, they join the party as a business associate if they handle that PHI for you. accountablehq
- If that vendor won't sign a business associate agreement (BAA), you don't have a provider. You have a liability. signeasy
This is where teams get “clever”:
- “We don't store it.”
- “We just pass it through.”
- “We redact the names so it’s fine.”
HIPAA cares about whether the data can identify a person and relates to their health, not whether you intended to do anything with it. If the payload can still link to an individual, you’re sending PHI to a third party. Full stop. ncbi.nlm.nih
Builder controls that actually matter
1) BAAs with any vendor that can see PHI
If an AI vendor can see PHI in prompts, files, embeddings, fine-tune sets, or tool outputs, you need a BAA. Not “after we get a pilot,” not “once sales closes.” Before you touch real users. hipaavault
Count the boring stuff too:
- logging and tracing vendors
- observability dashboards
- support tools that capture “screenshots” or request bodies
- AI gateways and proxy layers
If it can see prompts or responses, it can see PHI. metomic
2) Encryption, access control, and unexciting hosting choices
Do the basics, but treat them like production features, not a compliance checkbox. hhs
- TLS everywhere, including internal service calls.
- Encrypt at rest for databases, object storage, and backups.
- Role-based access with least privilege to ePHI. niu
- Audit trails for who accessed what and when. hhs
- HIPAA-aligned hosting and backup/replica configs that follow the same rules. hipaajournal
If staging is “prod with weaker controls and some real data,” what you actually built is a compliance bypass.
3) Redact PHI in prompts unless the provider is under BAA
No BAA? Treat that model endpoint like a public internet API. That means no PHI. accountablehq
Redaction needs to be concrete and deterministic to start. You can layer fancy stuff later. Something like this is a step up from nothing:
import re
PHI_PATTERNS = [
(re.compile(r"\b\d{3}-\d{2}-\d{4}\b"), "[SSN]"),
(re.compile(r"\b\d{10}\b"), "[PHONE]"),
(re.compile(r"\b\d{1,2}/\d{1,2}/\d{2,4}\b"), "[DATE]"),
]
def redact(text: str) -> str:
for pat, repl in PHI_PATTERNS:
text = pat.sub(repl, text)
return text
If you keep finding cases where you “absolutely need” the raw text, that's your signal to stop pretending: get the BAA, and design the system like it handles ePHI, because it does. hipaajournal
4) Vendor settings: training, retention, logging
Most AI providers now expose some knobs: accountablehq
- disable training on your data
- shorten data retention windows
- limit or disable prompt/content logging
Turn those on. Get them in the contract. Then verify they behave the way the slide deck claimed. Provider defaults are not tuned to your risk profile; they're tuned to whatever made sense for their product roadmap. signeasy
Chatbot guardrails that save you pain
A few rules that catch a lot of issues early:
- No PHI in system prompts. System prompts get copied around, cached, and reused. Put policy, instructions, and behavior there, not “Jane Doe had a stroke last month…”.
- Scope context per patient or encounter. One mega-thread “clinic assistant” that sees every patient is a great way to leak PHI across users through retrieval and memory.
- Match retention to your PHI lifecycle. If clinical records expire or roll off after X years, chat logs and embeddings that include PHI should follow the same lifecycle. hhs
Agents add another failure mode: tool calls. If the model can call search_patient_records and send_email, the tool layer needs hard permission checks and user scoping; the model is not a security boundary. Treat tools like untrusted input generators, not trusted peers. niu
What to log (and what to skip)
Logging is where “we care a lot about privacy” quietly becomes “we accidentally built a PHI archive.”
Log:
- request IDs, timestamps, model name/version
- token counts, latency, error codes
- hashes of prompts/responses so you can debug patterns without storing raw PHI
- redaction stats like “3 PHI matches removed”
Avoid:
- raw prompts/responses that contain PHI
- long-lived eval sets with real patient text
- “temporary” debug logs that never actually get deleted
If you truly need to inspect content, build a gated workflow: short retention, explicit access control, and a clear paper trail. metomic
Quick vendor sanity checklist
Use this as a quick gut check before wiring a new AI service into anything clinical: hipaavault
- [ ] BAA signed and actually covers subcontractors.
- [ ] Data residency matches your legal and contractual obligations.
- [ ] Training turned off on your data, retention written down.
- [ ] Prompt/content logging settings understood and configured.
- [ ] You can delete data on request and know how to trigger it.
- [ ] Your app can still function with PHI redacted for non-BAA providers.
HIPAA is mostly a pile of small decisions. Stack them well and you get reasonable safeguards. Stack them lazily and one day your “AI assistant” shows up as a line item in someone’s incident report. hipaajournal
-Sethers
