sethserver.com Subscribe
A colorful illustration of a robot-like figure with a transparent dome head containing stacked documents in cyan, orange, and magenta, standing on legs with wheels, surrounded by database icons, arrows, charts, and geometric shapes on a pale yellow background. Purple, orange, and cyan liquid appears to be dripping from the figure onto circular puddles on the ground below.

RAG, Vector DBs, and Leaky Knowledge Bases

By Seth Black • Updated: April 14, 2026

Security · 4 min read

Like this kind of writing? Get one email a week with notes on startups, AI, and the occasional strong opinion about Python: subscribe to the newsletter.

I've watched a chatbot confidently answer a question with another customer's contract. Not "a similar template." Not "a redacted sample." The actual thing. Names, pricing, terms, signatures. Makes your stomach do a little backflip.

This is the kind of thing that should make us all pause and seriously consider the implications of what we're building.

The fastest way to leak data: "We'll add filtering later"

Most RAG leaks aren't some complex spy-novel stuff. They're simple: a missing filter, a global index, and a programmer who assumed "someone else will handle security."

If you build one vector index for everyone, you've already made your life harder. Now every query needs perfect access control. Every time. No exceptions. No "oops, forgot to pass the tenant_id in this one endpoint."

The failure modes are pretty predictable:

If you've ever worked at an early stage startup, you already know how this happens: people ship the fastest, cheapest thing that looks like it works. Then they tape "AI" on the side and pray nobody asks hard questions.

Where RAG leaks actually happen

1) Retrieval before authorization

This is the classic: you do a similarity search, get top‑K chunks, then apply ACL checks. That's backwards.

If the retrieval step can see cross‑tenant content, you've already created a breach surface. Logs, traces, prompt stuffing, "helpful" error messages. Pick your poison.

2) You embedded stuff you never meant to search

Teams accidentally index:

3) The vector DB becomes a shadow database

If you store raw chunks alongside embeddings, you've built a second document store. Usually with worse access controls and fewer audits. Congrats.

A secure RAG architecture

You have two sane options:

Option A: Per-tenant indexes

This is the simplest mental model. One tenant, one index (or one namespace with hard isolation guarantees). It costs more. It's annoying to manage. It also prevents entire classes of mistakes. This is good for heavily regulated - read: big fines - industries like banking and healthcare.

Option B: Multi-tenant index with strict ACL filters

This can work, but only if your vector search supports pre-filtering by metadata and you treat that filter as mandatory.

Layer your metadata like you're paranoid (because you should be):

Then actually enforce it. Every time.

Ingestion-time controls: stop indexing garbage

Ingestion is where you win or lose.

Build a pipeline. File detection, text extraction, PII scan and redaction, classification, tagging, chunking, embedding, then write to the index. If that sounds tedious, it is. Do it anyway.

1) file type detection
2) text extraction
3) PII scan + redaction
4) classification + tagging
5) chunking
6) embedding
7) index write

Here's a sketch of what "tag everything" looks like:

doc = {
  "id": doc_id,
  "tenant_id": tenant_id,
  "department": "legal",
  "doc_type": "contract",
  "sensitivity": "restricted",
  "text": redacted_text,
}
vector_db.upsert(doc, embedding=embed(redacted_text))

Query-time controls: filters first, then similarity

The rule: authorization before retrieval.

Observability: log what was retrieved, not just what was answered

If you can't answer "which documents did we retrieve for this user and why," you're flying blind.

Log the user, tenant, query, retrieved doc IDs with metadata, scores, and which filters you applied.

Then monitor for:

Testing: try to break it on purpose

You need systematic queries that attempt cross-tenant retrieval. Do this frequently.

Write scripts that:

If you're building multi-tenant AI systems, isolating tools, agents, and tenants applies the same blast-radius thinking to agent actions. And if you're handling regulated data, check out HIPAA compliance for AI applications or GDPR data minimization for agents for practical controls.

RAG is not magic. It's a search system glued to a text generator. If your search system is sloppy, your generator will happily narrate the breach back to the user in perfect grammar.

-Sethers

Share this post
Newsletter

One email, once a week.

Notes on databases, systems, and the occasional strong opinion about Python. No spam, unsubscribe anytime.

Seth Black
Written by

Seth Black

Engineer and founder based in Texas. Writes about databases, AI, and running things in production. Embeds in small teams as lead engineer.

More from Security

View all →
Security

My Password Generators Had 53 Bits of Entropy

Sep 21, 2026
Security

API Key Generator with Prefix, Checksum and SHA-256

Sep 17, 2026
Security

Bcrypt Generator: Hash and Verify Passwords in Your Browser

Sep 17, 2026