New: the Agent Incident Ledger: disclosed incidents of AI gone rogueExplore the ledger
A red cereal box labelled "Security In the Loop" stands beside a bowl of rainbow cereal loops, while two robots share breakfast at a small table, the larger one spooning cereal for the smaller one.

Security in the Loop, Not in the Way: Guiding What AI Agents Do

Nov 5, 20266 Min

In your org, a developer is about to open a pull request, but their coding agent decides to publish its working plan as a public GitHub gist first, internal file paths included. A few days later, another agent testing how a webhook works in your code sends a file of real customer records to an endpoint nobody on the team has heard of. Neither agent was attacked. Rather, each one was trying to finish the task it was given, and took the shortest path it could find.

The first problem is finding out. Nothing failed and nobody was hacked, so nothing raised an alarm. At many companies this is probably happening already, and nobody knows. If you're lucky, someone spots the gist or the strange endpoint later. Once you do, your first move as a security engineer is to make sure it can't happen again. So you open a tool you already have, like an MCP gateway, and write a rule. Block that endpoint. Block that kind of upload. Done.

Every block rule has a cost

A block rule written after one incident is aimed at the last thing that went wrong. It often catches more than that, including work that was fine.

Each time it does, someone's task stalls. They have to notice, work out which rule fired, and ask security for an exception. Security then has to decide whether the rule was too broad. And that happens again every time the rule blocks something it shouldn't.

Agents add a cost of their own. When an agent hits a block it doesn't understand, it doesn't always stop. It may try a workaround or make up an explanation, and the rest of the session gets less useful.

A few rounds of it and the rule gets loosened, or quietly turned off, and you're back where you started.

Before you write the next block rule, ask whether you need to block at all.

Guide the agent, not the person

Unlike a person, an agent will take new instructions in the middle of a task. If you tell a person "don't do that, do this instead" while they're working, you've interrupted them. If you tell an agent the same thing, it can take the safer path and keep going.

That changes the job of a security policy. It can guide the agent the moment it acts.

This is how we think about security for agents at Manifold. We want security to know what happened, and also to stay in the loop and guide agents toward secure approaches across the whole organization. That covers every team running agents, not just developers.

Blocking still has its place, as one response among several that you choose on purpose.

The range of responses

A Manifold policy can respond to a risky agent action in several ways. Here they are from lightest to strictest. There's also Disable, which switches a policy off.

Response Does the action go ahead? Is a person interrupted? What the agent gets Use it for
Alert Does the action go ahead?Yes Is a person interrupted?No What the agent getsNothing Use it forSeeing how often a rule fires before you enforce it
Advise Does the action go ahead?Yes Is a person interrupted?No What the agent getsThe better way, for next time Use it forLow-risk habits you want to change over time
Correct Does the action go ahead?No Is a person interrupted?No What the agent getsThe safe way to do it Use it forRisky actions with a clear approved alternative
Ask Does the action go ahead?If a person approves Is a person interrupted?Yes What the agent getsA wait for the decision Use it forCalls a person should make
Block Does the action go ahead?No Is a person interrupted?Yes, they fix the session to carry on What the agent getsA refusal Use it forActions that should never happen
Block session Does the action go ahead?No Is a person interrupted?Yes, they start a new session What the agent getsThe session stops Use it forSigns the agent can’t be trusted anymore

Ask is for the calls a person should make, and Neal covered it in his post on Ask. This post is about the cases where nobody needs to decide. With Advise and Correct, no one has to step in, and the agent still ends up on the safe path.

Example: An agent writes a script that uploads log files to a public gist

Northwind and Larkspur are fictional companies.

Manifold
Correct mode · walkthrough

An agent routes around a corporate GitHub control to post a public gist

An engineer at Northwind asks their coding agent to get a customer's logs to the integrations team. Northwind's GitHub MCP server refuses to post them as a public gist, so the agent switches to the GitHub CLI, which the MCP's rules never see. Here's how Manifold caught the second route and steered the agent to a safe one without stopping the work.

01 · Background

A routine ask: get the customer's logs to the people who can fix it

Northwind engineer
Coding agent session · today
EngineerPull the last 24 hours of webhook logs from Larkspur's production tenant and get them to the integrations team. Deliveries are failing and they need to debug it.
10:41:52Fetchtenant=larkspur-prod · webhooks · last 24h
10:42:01Write/tmp/larkspur-webhooks.log24h of failures

Nothing wrong so far. The data is on the engineer's laptop. Now the agent has to get it to the integrations team.

1 / 9 click a step or use the arrows · hover to hold
02 · First try: the corporate GitHub MCP

Northwind's GitHub MCP refuses to post a public gist

The agent wanted a gist link the team could open from Slack. Its first stop was the GitHub MCP server that Northwind's IT team manages, which is set up to refuse public gists.

10:42:03MCPgithub · create_gist {"public":true,"files":{"larkspur-webhooks.log":…}}corporate MCP
github MCPError: public gists are disabled by Northwind's GitHub policy.

The control did its job. But the error only says no. It doesn't say why, or what to do instead, so the agent treats it as a tooling problem to solve.

2 / 9 click a step or use the arrows · hover to hold
03 · The workaround

Same gist, different door

AgentThe GitHub MCP won't create public gists. The gh CLI is installed and logged in, so I'll use that instead.
bash · agent tool calls
$ gh auth status github.com ✓ Logged in to github.com account northwind-engineer (keyring) - Token scopes: 'gist', 'read:org', 'repo', 'workflow' $ gh gist create --public \anyone can read this --desc "larkspur webhook failures" \ /tmp/larkspur-webhooks.log
Door 1: Northwind's GitHub MCPCorporate policy applies. Public gist refused.
refused
Door 2: the gh CLIThe developer's own login, straight to api.github.com. The MCP never sees it.
open
3 / 9 click a step or use the arrows · hover to hold
04 · The issue

The log is Larkspur's data, not Northwind's

Every failed delivery was logged with its full payload: order events for Larkspur's own customers. If the command had run, Northwind would have published that personal data under the developer's GitHub account.

/tmp/larkspur-webhooks.log
03:14:07Z order.created status=500 attempt=3
  customer.name="Jane Doe"
  customer.email="jane.doe@example.com"
  customer.phone="+1 415 555 0142"
03:14:09Z order.created status=500 attempt=1
  customer.name="John Smith"
  customer.email="john.smith@example.net"
  … 24 hours of failures
Public gistReadable by anyone with or without the link
GitHub's public feedListed alongside every other public gist
Crawled by botsScraped for leaked data
Breach noticeThe kind of incident it ends with
4 / 9 click a step or use the arrows · hover to hold
05 · Why a block rule would hurt

The MCP rule was right. It just only covers the MCP.

The obvious ways to close the second door are all too blunt. Each one stops this gist, and each one stops a lot of legitimate work too.

Block
the gh CLI
What breaks

Pull requests, issues, CI runs and releases. For most coding agents, gh is the main way they work with GitHub.

Block
api.github.com
What breaks

Pushes, pull requests, CI checks, and the corporate GitHub MCP itself, which calls the same API.

Block
gists entirely
What breaks

Nothing gets fixed. The team still needs the log, so the problem just moves to the next tool that can post it.

The goal isn't to take tools away. It's to apply the same rule on every route and point the agent at a safe one.
5 / 9 click a step or use the arrows · hover to hold
06 · The policy that matched

Correct Public Gist

Correct
Denies
Public gists by any route: the gh CLI, the GitHub API or the GitHub MCP
Returns
The safe form, for the agent to retry
policy.cedar
forbid(principal, action, resource is Event::Tool) when { (resource.input like "*api.github.com/gists*" && ( resource.input like "*public*: True*" || resource.input like "*public*:True*" || resource.input like "*public*: true*" || resource.input like "*public*:true*" ) ) || resource.input like "*gh gist create*--public*" ||matched resource.input like "*gh gist create* -p*" || ( resource.server == "github" && resource.name == "create_gist" && resource.input like "*\"public\":true*" ) };

Seen before anything ran

The full gh command, including --public
The file it would upload: customer tenant logs
The task the agent was given
Where the file came from: Larkspur's production tenant
The GitHub MCP refusal three seconds earlier, in the same session
6 / 9 click a step or use the arrows · hover to hold
07 · What the agent received

Denied, with the reason and the safe route spelled out

10:42:06Bashgh gist create --public --desc "larkspur webhook failures" /tmp/larkspur-webhooks.log--public
Manifold · CorrectDenied. Customer tenant data can't be posted to public gists by any route, including the gh CLI (Northwind Data Handling Policy). Put the file on the internal file share at /Volumes/eng-share/ instead. Share the path, not the contents.
10:42:06Resultgist not created · no connection to GitHubcall never ran

Only this one call was stopped. Unlike the MCP's bare error, the agent now knows why, and it has a route that works.

7 / 9 click a step or use the arrows · hover to hold
08 · What the agent did next

It stopped looking for another door and took the safe route

10:42:06ManifoldCorrect · public gist via gh CLI denied
10:42:09Bashmkdir -p /Volumes/eng-share/incidents/larkspur-webhooks/
10:42:09Bashcp /tmp/larkspur-webhooks.log /Volumes/eng-share/incidents/larkspur-webhooks/internal share
10:42:12Slack#integrations · posted the file's pathpath, not contents
10:42:14ReplyTo the engineer: why the gist was stopped and where the log went
Policy didn't fire again Team can start debugging Customer data never left Northwind's systems
8 / 9 click a step or use the arrows · hover to hold
09 · What security saw

One finding, already resolved

today · 10:42
Correct Correct Public Gist matched MCP bypass Resolved
Who and whereThe engineer, their laptop and the coding agent
Commandgh gist create --public …, run with the developer's own gh login
EarlierNorthwind's GitHub MCP refused the same public gist at 10:42:03
Data/tmp/larkspur-webhooks.log, from Larkspur's production tenant
DestinationA public gist, never created
PolicyCorrect Public Gist · Correct mode · Northwind Data Handling Policy
MessageThe denial message, word for word
What came nextThe copy on the internal file share: /Volumes/eng-share/incidents/larkspur-webhooks/
9 / 9 click a step or use the arrows · hover to hold
HTML slideshow for the fictional example.

Background

An engineer at Northwind, a SaaS company, asked their coding agent to pull the logs for Larkspur, one of Northwind's customers, and get them to the integrations team to troubleshoot an issue. The agent pulled the last 24 hours of webhook logs from Larkspur's production tenant into /tmp/larkspur-webhooks.log, and it needed to get them to the integrations team so they could debug the failures.

What the agent tried

The agent decided the quickest way to share the log was a GitHub gist link the team could open from Slack. The agent tries to create the Gist via the corporate Github MCP but is blocked because the MCP does not allow Gist. So to bypass this the agent ran the github cli directly:

gh gist create --public --desc "larkspur webhook failures" /tmp/larkspur-webhooks.log

The issue

Because the agent leveraged the Github CLI, it is able to bypass this restriction. --public makes the gist readable by anyone and lists it in GitHub's public feed, which bots crawl for leaked data. The log was Larkspur's data, not something that should be exposed publicly. Every failed delivery was logged with its full payload, and those payloads were order events for Larkspur's own customers: names, email addresses, phone numbers and shipping addresses. If the script had run, Northwind would have published personal data belonging to Larkspur's customers under the developer’s GitHub account.

Why a block rule would hurt

The obvious rules are all too blunt. Users and Agents have good reasons to call the Github CLI, so blocking this utility isn’t an option. Blocking api.github.com breaks pushes, pull requests, CI checks and half of what the agent was set up to do. Even blocking gists entirely only moves the problem.

The policy that matched

Correct Public Gist, running in Correct mode. It denies scripts and commands that send file contents to public destinations, such as public gists and paste sites, and returns the safe form for the agent to retry.

Manifold catches this where EDR can't because it sees the tool call itself, before anything runs: the full text of the script, including the destination and the public flag, the task the agent was given and where the file came from.

forbid(principal, action, resource is Event::Tool) when { (resource.input like "*api.github.com/gists*" && ( resource.input like "*public*: True*" || resource.input like "*public*:True*" || resource.input like "*public*: true*" || resource.input like "*public*:true*" ) ) || resource.input like "*gh gist create*--public*" || resource.input like "*gh gist create* -p*" || ( resource.server == "github" && resource.name == "create_gist" && resource.input like "*\"public\":true*" ) };

What the agent received

The policy stopped the script before it ran, so the gist was never created. Instead of a bare refusal, the agent got this message back:

Denied. Customer tenant data can't be posted to public gists or paste sites, which are crawled within minutes (Company Data Handling Policy). Put the file on the internal file share at /Volumes/eng-share/ instead. Share the path, not the contents.

What the agent did next

It dropped the gist and copied the customer webhook log to the internal file share, in a folder for the incident. The policy didn't fire again. The agent posted the file's path in the integrations team's Slack channel, and the team had what they needed to start debugging. Larkspur's data never left Northwind. The agent's summary to the developer said why the gist had been stopped and where the log went.

What security saw

The console recorded one finding, already resolved. It showed:

  • who and where: the developer, their laptop and the agent
  • the full script the agent tried to run, which EDR never had
  • the customer log it would have read, and the public gist it would have created
  • the policy that matched, Correct mode, and the standard it enforces
  • the denial message the agent received
  • what the agent did next: the copy it put on the internal file share

When you do need to stop

Sometimes guiding the agent isn't enough, because the agent itself can't be trusted anymore. Two signs of that are a suspected prompt injection and repeated attempts to reach credentials.

That's what Block session is for. It stops the risky action and everything after it in that session. To carry on, the person working with the agent starts a new session.

Keep this for a small set of strong signals. Used everywhere, it brings back the same cost as a block rule.

What security sees, whatever the response

Every response except Disable records a violation, so security gets one history across all of them. A rule set to Advise shows up in the same place as one set to Block session. You can see what agents tried, how often, and which rules are doing the most work.

That's what keeps security in the loop even when nobody was interrupted.

Where to start

Start by finding your use cases. Make a list of what you block today, in any tool. For each item, ask two questions:

  • Is there a safe way to do this that the agent could be pointed to instead? If so, try Correct or Advise.
  • Is this a call a person should make? If so, try Ask.

What's left after that is your real block list. It will probably be shorter than the one you started with, and every rule on it will be one you meant to keep.

Most of the time, the agent corrects course on its own, and nobody's work stops.

About the author

  • Oleksandr Yaremchuk

    Oleksandr Yaremchuk

    CTO & Co-Founder

    Oleksandr is an engineering leader with 15+ years of building and scaling teams at companies of all sizes. He co-created LLM Guard, an open-source LLM firewall with over 12 million downloads, along with the security models that power it.