sethserver.com Subscribe
A purple and white robot sits at a control panel desk holding a magnifying glass, surrounded by floating warning icons including yellow triangles with exclamation marks, bells, and speech bubbles labeled "ALERT," "WARNING," and "ERROR" against a turquoise background. The robot has dual circular displays showing colorful graphs on its face, and the control panel features various buttons, screens, and pie chart displays.

Logging, Monitoring, and Incident Response for AI Systems

By Seth Black • Updated: April 14, 2026

Security · 5 min read

Like this kind of writing? Get one email a week with notes on startups, AI, and the occasional strong opinion about Python: subscribe to the newsletter.

I learned the word hallucination the fun way: watching a model confidently invent a fact that never existed. My first thought was not deep or thoughtful. It was: "what the crap?" The second thought was, "okay, show me exactly how you got there." That's where the real problem showed up.

No logs. No trace. Just vibes.

If you're running AI in production and your logging story is "we log errors," you're toast. AI incidents are almost never a nice clean exception with a stack trace. They're "the agent did something weird," and now you have to reconstruct a conversation, a tool call, a model version, and a bunch of hidden context that quietly changed underneath you.

First rule: log the workflow, not just the API call

An AI request is a pipeline. Even a “simple” chat turns into a chain:

If you can't replay that chain, you can't debug it.

The minimum I want for each step:

If you have tenants, log tenant. If you have environments, log environment. If you have users, log a user identity or a stable hash. Otherwise you'll stare at a spike on a dashboard thinking, "Cool. Which customer is angry?"

Correlation IDs: the boring glue that saves your weekend

Give every “AI run” a run_id. Every tool call gets a tool_call_id. Every downstream service call carries those IDs. It's boring plumbing, and it's the difference between knowing and guessing.

A simple structured log shape:

{
  "ts": "2026-04-14T18:22:11Z",
  "env": "prod",
  "tenant_id": "t_123",
  "user_id": "u_456",
  "run_id": "run_9c1f",
  "step": "tool_call",
  "tool_name": "crm.search",
  "model": "gpt-4.1-mini",
  "prompt_version": "agent_v17",
  "latency_ms": 842,
  "tokens_in": 3120,
  "tokens_out": 220,
  "redaction": "pii_v3",
  "payload": {
    "query": "[REDACTED_EMAIL]"
  }
}

You don't have to start fancier than this. Just make sure everything passes the same IDs around.

Redaction and minimization: logs are a second data leak waiting to happen

I worked at a SaaS shop that advertised “AI automation,” and then I found out a big chunk of it was spreadsheets, formatting, and humans doing the work on the side. That mess made me paranoid in a useful way: if your process is sloppy, your data boundaries are sloppy too.

Logging can quietly turn into a shadow database of secrets.

Rules that actually hold up:

And yes, you need to test your redaction. Not by reading the code and nodding. By actively trying to punch holes in it.

Monitoring the AI stack: watch behavior, not just uptime

Traditional monitoring asks: "Is the service up?" AI monitoring asks: "Is the service behaving like itself?"

Metrics I like:

Dashboards should slice by:

Alerts that matter: "weird but safe" vs "call someone now"

Most alert setups are noise because they’re built for exceptions, not for “the AI lost the plot.”

Signals for possible abuse:

Signals for “wake a human up”:

My favorite filter is simple: alert on deltas. If a tenant's tool usage jumps 10x in 10 minutes, that's not a fun traffic pattern. That's “go look right now.”

Incident response for AI: isolate, disable, rollback

AI incidents need playbooks that assume the model is a chaotic intern with root access.

Triage steps that tend to work:

  1. Freeze the evidence: capture the run_id traces and related tool logs.
  2. Isolate the blast radius: disable or throttle the risky tool, not the whole product.
  3. Rollback prompt/model versions: treat prompts like deployable artifacts with versions and changelogs.
  4. Add a temporary guardrail: block a tool call pattern, enforce allow-lists, clamp arguments.
  5. Reproduce with a replay harness: same prompt, same retrieved docs, same tool stubs.

Playbooks I keep nearby:

The principle of least privilege in the AI age means scoping every tool to the minimum permissions you can live with—because hallucinations turn from goofy chat errors into expensive, real-world damage the moment agents get broad access.

Post-mortems that actually harden the system

A post-mortem only matters if it changes code and tests, not just slides.

Feed the lessons back into:

Treat incidents like new unit tests. If you can't simulate the failure on demand, you didn't really fix it.

Production incidents don't care about your diagrams. They care about whether you can reconstruct what happened, quickly, without guessing. Logging the whole workflow is how you stop guessing.

-Sethers

Share this post
Newsletter

One email, once a week.

Notes on databases, systems, and the occasional strong opinion about Python. No spam, unsubscribe anytime.

Seth Black
Written by

Seth Black

Engineer and founder based in Texas. Writes about databases, AI, and running things in production. Embeds in small teams as lead engineer.

More from Security

View all →
Security

My Password Generators Had 53 Bits of Entropy

Sep 21, 2026
Security

API Key Generator with Prefix, Checksum and SHA-256

Sep 17, 2026
Security

Bcrypt Generator: Hash and Verify Passwords in Your Browser

Sep 17, 2026