sethserver.com Subscribe
A blue robot with a simple smiling face holds a pink scrub brush with soap bubbles, standing next to a yellow shield with a padlock symbol. Surrounding the robot are floating credit cards (purple, coral, yellow, and teal including one labeled "SOCIAL SECURITY") and white paper documents against a light blue-green background with sparkle effects.

Building an LLM Proxy That Scrubs SSNs, Credit Cards, and PII Before It Leaks

By Seth Black • Updated: September 17, 2026

AI · 7 min read

Like this kind of writing? Get one email a week with notes on startups, AI, and the occasional strong opinion about Python: subscribe to the newsletter.

The mistake: trusting the model to “behave”

The first time I tried to use generated personas in a survey system, I gave the model a dead-simple setup: “You’re a janitor, you make 15k a year.” Then I lobbed the easiest question possible: “What’s your job title?”

It replied: “CEO.”

That was the whole lesson in one response. An LLM is not “role-playing” in any meaningful, trustworthy way. It is trying to spit out likely tokens in the bucket labeled “job title,” and both “janitor” and “CEO” live in that bucket. It is not lying, and it is definitely not “respecting your persona system.” It is just predicting text. arxiv

If you hook that thing up to real user data and assume persona prompts and fine-tuned vibes will enforce your security and compliance rules, you’re building a fence out of wet paper.

You need something in front of the model whose only job is: “Should this request be allowed, and under what constraints?” That is the LLM proxy.


The architecture pattern: client → proxy → model (and tools)

The basic shape looks like this:

The proxy is where you answer questions like:

If you skip the proxy, you’re effectively letting “CEO Janitor” run your security program. The model happily mixes “dump the database” and “answer the user’s question” in the same space of “things that sound helpful.” petronellatech


What the proxy actually does

1. Input classification

First step is tagging the request. It doesn’t have to be perfect to be useful.

You want tags like:

Even a naive classifier or a couple of lookups from your auth context is better than nothing. Once you have tags, you can write actual policy instead of giant if statements scattered across services.

2. Catch the obvious stuff with rules

Regex and simple rules get you surprisingly far:

These are fast, cheap, and easy to explain. When legal or security asks “Why did we block this?”, you can point at the exact pattern instead of shrugging at a model output.

3. Fuzzy PII detection with ML / LLM (optional)

Once you outgrow rules, you can layer on a classifier. That can be:

Do this before the main model call and, if you care about data boundaries, don’t use the same external provider you’re trying to keep data away from. Otherwise your “privacy filter” is just another path where raw data leaks out while you congratulate yourself on being compliant. datasunrise

4. Redaction strategies

You’ve basically got three knobs:

Most real systems end up mixing all three. Cards might get tokenized, SSNs might be hard-blocked, emails might be masked except on internal routes.

5. Observability and audit tags

You want to log the fact that sensitive data was there, not the data itself.

Example audit record:

This makes incident reviews bearable. You can actually answer “Did this leave our boundary? What rule fired?” instead of playing forensic guessing games across log streams.


Where to enforce: three choke points

You get three main choke points:

  1. Incoming prompts (user → model).
  2. Tool parameters (model → tools).
  3. Outgoing responses (tools → model → user). startupsoft

People usually nail number one and kind of remember number three. Number two is where the fire starts.

If you give the model generic access to tools, sooner or later it will happily attempt:

Not because it’s evil. Because that looks like a complete, helpful solution according to the training data. The model has no concept of “scope” unless you give it one with real guardrails. arxiv

This is where the old principle of least privilege shows up in a new outfit: tools should only have the minimum permissions needed to do the thing they’re meant to do, scoped to the current user, tenant, and context. If a “view invoice” tool can also quietly enumerate the entire customer table, you don’t have AI agents, you have a breach generator. nightfall


A concrete code walk: FastAPI middleware scrubbing JSON

Here’s a trimmed-down example of a JSON-scrubbing middleware in FastAPI. The point is the shape, not the exact framework details. fastapi.tiangolo

import re
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse

SSN_RE = re.compile(r"\b\d{3}-\d{2}-\d{4}\b")
EMAIL_RE = re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b")

def mask_ssn(text: str) -> str:
    return SSN_RE.sub("***-**-****", text)

def contains_pii(text: str) -> bool:
    return bool(SSN_RE.search(text) or EMAIL_RE.search(text))

app = FastAPI()

@app.middleware("http")
async def pii_scrubber(request: Request, call_next):
    body = await request.body()
    text = body.decode("utf-8", errors="ignore")

    tags = {
        "contains_pii": contains_pii(text),
        # Put tenant / jurisdiction here based on headers or auth context
    }

    # Example policy: block SSNs for external models
    if SSN_RE.search(text) and request.headers.get("x-model-class") == "external":
        return JSONResponse(
            status_code=400,
            content={"error": "SSN detected. Request blocked.", "tags": tags},
        )

    # Example policy: redact SSNs on a chat route
    if request.url.path.startswith("/chat"):
        text = mask_ssn(text)
        # Here you’d rebuild the request with the scrubbed body

    request.state.policy_tags = tags
    response = await call_next(request)
    response.headers["x-policy-tags"] = str(tags)
    return response

Rebuilding the request object with the modified body is annoying and slightly framework-specific. You’ll need to read the docs or wrap your own request class. But once you’ve got this in place, you have a single, testable place to enforce rules before anything reaches the model. fastapi.tiangolo


Why this matters even more with “AI glue”

I’ve worked at a place that marketed “AI automation” and the core engine was spreadsheet column-mapping plus some string cleaning. That wasn’t a scam. That was the job. Glue code with opinions. datasunrise

LLM-based glue is the same thing, except now the glue can:

If you bolt that onto production without a proxy, you’re not “shipping fast,” you’re hand-delivering your risk register to an autocomplete engine.

Put a proxy in front of the model. Scrub inputs. Scope and gate tool calls. Tag everything. Treat Model Context Protocol or any other tool-host system as behind a policy layer, not as the policy layer itself. docs.litellm

The model is great at predicting the next token. It is not your compliance team. It is not your security boundary. And if you ask it who’s in charge, it might still say “CEO Janitor.”

– Sethers

Share this post
Newsletter

One email, once a week.

Notes on databases, systems, and the occasional strong opinion about Python. No spam, unsubscribe anytime.

Seth Black
Written by

Seth Black

Engineer and founder based in Texas. Writes about databases, AI, and running things in production. Embeds in small teams as lead engineer.

More from AI

View all →
AI

How to be a Good Farrier: What the Car Teaches Us About AI and Your Job

Nov 10, 2026
AI

Principle of Least Privilege in the AI Age

Apr 14, 2026
AI

How to Actually Secure MCP (Model Context Protocol)

Apr 14, 2026