You can learn a lot about an LLM by watching it lie to your face without missing a beat. linkedin
I built an AI survey response system where "generated personas" answered questions based on their attributes. The persona would say something like: job title = janitor, salary = $15,000/year. Then I'd ask, "What's your job title?" and it would answer, "CEO." Salary? "$35,000/year." Fully fabricated, zero shame. ibm
That's when it really clicked: an LLM is not a smart intern. It's a next-token machine that's very good at sounding relaxed while being wrong. "Janitor" and "CEO" both live in the same mental neighborhood around "job," so the model just hops the fence and keeps talking. arxiv
Now take that same thing and bolt on tools. A database. An email sender. A billing API. You've wired a sometimes-wrong text generator directly into your infrastructure. datarobot
That's why the Principle of Least Privilege matters more with AI agents than it ever did in a boring CRUD app. csrc.nist
Tools turn weird output into real damage
Plain chat hallucinations are annoying. Tool-using hallucinations are expensive and persistent. linkedin
Agents don't just answer once; they plan and chain actions together. A tiny mistake turns into a little Rube Goldberg machine of regret: arxiv
- Fetch the wrong customer record.
- Decide it's correct because it "looks right."
- Update it.
- Email the customer about the change.
- Log a "success" event so nobody realizes anything broke. linkedin
Nobody sat down and tried to design a breach. The system simply did everything it was allowed to do. cisecurity
Least privilege in security says each user or process gets only the minimum access needed to do its job. With agents, I'd add: don't give them more verbs than they need either. crowdstrike
Translate security basics into AI terms
Security folks have already done most of this thinking. The twist now is we're granting permissions to a system that will occasionally say "CEO" no matter what you wrote to disk. learn.microsoft
Identity: who is the agent right now?
At any given moment, your agent is acting as one of these:
- A real user (delegated access, "act on my behalf")
- A service account (pure system automation)
- A special "assistant" identity that lives somewhere in between csrc.nist
Pick one and be explicit. learn.microsoft
If the agent is acting for a user, give it only what that user has, and ideally a bit less. If it's a service, scope it to a single clear job instead of a generic all-powerful helper account. crowdstrike
Scope: tools, tenants, and data slices
This is where a lot of AI apps go off the rails. Someone builds one mega-tool like: datarobot
admin_tool(action, payload)
Now the model is holding a Swiss Army chainsaw. curity
Break tools up by intent instead: curity
read_invoice(invoice_id)list_invoices(org_id, limit)pay_invoice(invoice_id, amount)
Yes, it's more code. It's also how you stop the model from "getting creative" and inventing new behaviors inside a generic admin escape hatch. arxiv
Also, per-tenant scoping is not optional. cisecurity
If your tool takes org_id and user_id, do not let the model pick them. Pass them from the server-side session and ignore anything the model tries to override. snyk
Bad:
{ "org_id": "???", "query": "latest invoices" }
Better:
{ "org_id": "<from_session>", "query": "latest invoices" }
Cross-tenant access is where an innocent "summarize recent activity" turns into "why does this user see data from another company?" cisecurity
Time: sessions vs long-lived keys
Long-lived tokens are how you turn a one-off mistake into a slow-moving incident report. snyk
Agents should use short-lived, session-scoped credentials. If the agent is doing a task for a user, bind the token to: snyk
- That user
- That org
- That task
- That time window curity
If the agent can run for hours or days with the same keys, you've basically handed a wandering background process your house keys and hoped the logs catch it. learn.microsoft
Concrete rules I follow (because I like sleeping)
Here's the checklist I actually use now, stitched together from security best practices and a few hard lessons. okta
- Default read-only. Start with read scopes and add write later, the way you would with OAuth scopes or API permissions. csrc.nist
- Narrow tools. Prefer several focused tools over a single "do everything" endpoint. crowdstrike
- No cross-tenant reads. Ever. For support, build a separate, auditable support workflow instead of bending normal rules. cisecurity
- Server-side scoping.
org_id,user_id, and permissions come from the authenticated session or token, not from the prompt. snyk - Make destructive actions annoying. Require an explicit confirmation step, a second tool call, or higher-friction auth for high-risk operations. curity
The last one matters. Models are naturally good at marching forward; stopping mid-plan is not something they do on their own. datarobot
Ship it, then tighten it quickly
I like shipping early. I also have a long history of building systems that are a little too clever for their own good. At one point I wrote custom BI software in C++ and rolled my own encryption scheme so database credentials could hop machines without using an off-the-shelf solution. screenconnect
It worked. It was also completely unnecessary, and standard tooling would have done the job with less risk and less ego. okta
AI apps lean toward the opposite failure mode. You move fast, strap an agent to real systems, and tell yourself you'll "lock it down later." Later usually shows up right after your agent emails the wrong customer or pays the wrong invoice. ibm
The approach I actually use now, aligned with least-privilege and scope best practices: csrc.nist
- Ship the first usable version with read-only tools. crowdstrike
- Watch how real users and real data interact with the system. datarobot
- Add write tools one by one, each with tight scope and clear audit trails. learn.microsoft
- Tighten permissions as soon as you prove value instead of waiting for a scare. csrc.nist
Recovery beats perfection, but permissions are not a cosmetic improvement. They're the line between "funny chatbot story" and "why did this thing just create a real incident ticket for us?" ibm
-Sethers
