---
title: "RAG, Vector DBs, and Leaky Knowledge Bases"
description: "Learn how to prevent data leaks in RAG systems through proper tenant isolation, access controls, and ingestion pipelines. Covers secure architecture patterns, pre-filtering strategies, PII redaction, and testing methods to stop chatbots from exposing sensitive customer data across organizational boundaries."
image: "https://cdn.sethserver.com/blog-images/rag-vector-dbs-and-leaky-knowledge-bases-social-1794886865.jpg"
---

# RAG, Vector DBs, and Leaky Knowledge Bases

I've watched a chatbot confidently answer a question with *another customer's contract*. Not "a similar template." Not "a redacted sample." The actual thing. Names, pricing, terms, signatures. Makes your stomach do a little backflip.

This is the kind of thing that should make us all pause and seriously consider the implications of what we're building.

## The fastest way to leak data: "We'll add filtering later"

Most RAG leaks aren't some complex spy-novel stuff. They're simple: a missing filter, a global index, and a programmer who assumed "someone else will handle security."

If you build one vector index for everyone, you've already made your life harder. Now every query needs perfect access control. Every time. No exceptions. No "oops, forgot to pass the tenant\_id in this one endpoint."

The failure modes are pretty predictable:

- **Missing tenant filters**: similarity search returns "closest" chunks, even if they belong to another org.
- **Global indexes with weak ACL logic**: you filter *after* retrieval, which means sensitive chunks still touched the pipeline.
- **Shared embeddings store**: raw text sits next to vectors, and someone ships a debug endpoint that returns both.

If you've ever worked at an early stage startup, you already know how this happens: people ship the fastest, cheapest thing that looks like it works. Then they tape "AI" on the side and pray nobody asks hard questions.

## Where RAG leaks actually happen

### 1) Retrieval before authorization

This is the classic: you do a similarity search, get top‑K chunks, then apply ACL checks. That's backwards.

If the retrieval step can see cross‑tenant content, you've already created a breach surface. Logs, traces, prompt stuffing, "helpful" error messages. Pick your poison.

### 2) You embedded stuff you never meant to search

Teams accidentally index:

- Internal prompts ("system: you are a helpful agent... here's the admin password...")
- Config files
- API keys in README snippets
- Customer support exports with everything in them

### 3) The vector DB becomes a shadow database

If you store raw chunks alongside embeddings, you've built a second document store. Usually with worse access controls and fewer audits. Congrats.

## A secure RAG architecture

You have two sane options:

### Option A: **Per-tenant indexes**

This is the simplest mental model. One tenant, one index (or one namespace with hard isolation guarantees). It costs more. It's annoying to manage. It also prevents entire classes of mistakes. This is good for heavily regulated - read: big fines - industries like banking and healthcare.

### Option B: **Multi-tenant index with strict ACL filters**

This can work, but only if your vector search supports *pre-filtering* by metadata and you treat that filter as mandatory.

Layer your metadata like you're paranoid (because you should be):

- `tenant_id`
- `department`
- `doc_type`
- `sensitivity` (public / internal / confidential / restricted)
- `source_system`
- `retention_policy`

Then actually enforce it. Every time.

## Ingestion-time controls: stop indexing garbage

Ingestion is where you win or lose.

- **Classify and tag** every doc before it hits the index.
- **Redact or pseudonymize** where you can.
- **Don't embed SSNs or unique identifiers unless you need them. You probably don't.**

Build a pipeline. File detection, text extraction, PII scan and redaction, classification, tagging, chunking, embedding, then write to the index. If that sounds tedious, it is. Do it anyway.

1) file type detection
2) text extraction
3) PII scan + redaction
4) classification + tagging
5) chunking
6) embedding
7) index write

Here's a sketch of what "tag everything" looks like:

```python
doc = {
  "id": doc_id,
  "tenant_id": tenant_id,
  "department": "legal",
  "doc_type": "contract",
  "sensitivity": "restricted",
  "text": redacted_text,
}
vector_db.upsert(doc, embedding=embed(redacted_text))
```

## Query-time controls: filters first, then similarity

The rule: **authorization before retrieval**.

- Enforce `tenant_id` and `sensitivity` filters in the vector query itself.
- Keep `top_k` small. Big K is how you "accidentally" pull in weird stuff.
- Add sanity checks on the returned chunks. If a chunk contains forbidden patterns (other tenant names, certain doc types), drop it.
- Some docs should be visible to humans but not to the model. That's normal. Make it policy.

## Observability: log what was retrieved, not just what was answered

If you can't answer "which documents did we retrieve for this user and why," you're flying blind.

Log the user, tenant, query, retrieved doc IDs with metadata, scores, and which filters you applied.

Then monitor for:

- retrieval from unexpected tenants (should be impossible)
- sensitivity mismatches (restricted doc retrieved for a basic user)
- sudden spikes in retrieval failures (often a sign someone "optimized" filters away)

## Testing: try to break it on purpose

You need systematic queries that attempt cross-tenant retrieval. Do this frequently.

Write scripts that:

- create two tenants with similar documents
- ask the same questions from both sides
- confirm retrieved doc IDs never cross the boundary
- scan the index for mis-tagged docs (missing tenant\_id is a red flag, not a warning)

If you're building multi-tenant AI systems, [isolating tools, agents, and tenants](https://www.sethserver.com/ai/isolating-tools-agents-and-tenants-so-one-prompt-cant-nuke-everything.html) applies the same blast-radius thinking to agent actions. And if you're handling regulated data, check out [HIPAA compliance for AI applications](https://www.sethserver.com/ai/hipaa-phi-and-ai-how-not-to-accidentally-become-a-noncompliant-data-processor.html) or [GDPR data minimization for agents](https://www.sethserver.com/ai/gdpr-eu-ai-act-and-your-ai-stack-data-minimization-for-agents.html) for practical controls.

RAG is not magic. It's a search system glued to a text generator. If your search system is sloppy, your generator will happily narrate the breach back to the user in perfect grammar.

-Sethers


```json
{"@context": "https://schema.org", "@type": "BlogPosting", "headline": "RAG, Vector DBs, and Leaky Knowledge Bases", "datePublished": "2026-04-14T19:02:54", "dateModified": "2026-04-14T19:02:54", "image": {"@type": "ImageObject", "url": "https://cdn.sethserver.com/blog-images/rag-vector-dbs-and-leaky-knowledge-bases-social-1794886865.jpg", "width": 400, "height": 400}, "url": "https://www.sethserver.com/security/rag-vector-dbs-and-leaky-knowledge-bases.html", "author": {"@type": "Person", "name": "Seth Black", "url": "https://www.linkedin.com/in/seth-black-tx"}, "publisher": {"@type": "Organization", "name": "Sethserver", "logo": {"@type": "ImageObject", "url": "https://www.sethserver.com/static/images/PaperAirplane512.png"}}, "description": "Learn how to prevent data leaks in RAG systems through proper tenant isolation, access controls, and ingestion pipelines. Covers secure architecture patterns, pre-filtering strategies, PII redaction, and testing methods to stop chatbots from exposing sensitive customer data across organizational boundaries.", "articleSection": "Technology"}
{"@context": "https://schema.org", "@type": "BreadcrumbList", "itemListElement": [{"@type": "ListItem", "position": 1, "name": "sethserver", "item": "https://www.sethserver.com/"}, {"@type": "ListItem", "position": 2, "name": "Security", "item": "https://www.sethserver.com/security/"}, {"@type": "ListItem", "position": 3, "name": "RAG, Vector DBs, and Leaky Knowledge Bases", "item": "https://www.sethserver.com/security/rag-vector-dbs-and-leaky-knowledge-bases.html"}]}
```
