When we started building Mode Agent, one of the first questions we had to answer was how to keep an AI agent safe when it is operating autonomously. Reading files, calling APIs, browsing the web, delegating to other agents. Real work, with real consequences if something goes wrong.
The obvious answer is the one most products use. Write safety instructions in the system prompt. Tell the model what it can and cannot do. Trust it to comply.
We started reading the research. It does not hold up.
The problem with prompt-based constraints
An AI agent's safety instructions live inside its context window. That sounds fine until you consider what else lives there: everything the agent reads, processes, and receives as tool output. Web pages it visits. Documents it opens. Responses from APIs it called.
Any of that content can contain adversarial instructions. Research published in 2026 found that intent-hiding attacks, which frame malicious instructions as credible content, achieve success rates of 85 to 100 percent against tested systems (arXiv 2603.04474). The model reads its safety rules. The model also reads the web page telling it to ignore those rules. It cannot reliably tell the difference.
This is not a model quality problem. All current frontier models share the same vulnerability. Prompt injection is an architectural issue, not a capability gap. A smarter model does not fix it.
There is a second problem that requires no adversarial action at all. As an agent session extends across dozens of tool calls and turns, the context window fills. Most frameworks handle this by compressing earlier content into summaries, truncating the oldest turns, or sliding a window that drops content past a length threshold. Safety instructions written at session start can be summarized away, pushed toward the middle of a long context where retrieval performance can degrade significantly, or dropped entirely. The rules are still written somewhere. The model may simply no longer have them.
What happens when one agent fails
In a single-agent setup, a constraint violation is contained. In a multi-agent workflow, it is not.
Mode Agent runs specialized agents in parallel across engineering, research, product, and more. When one agent produces output that violates a constraint, that output becomes input to the next agent. The violation propagates. Research on multi-agent error cascades found that five of six tested agent frameworks reach 100% system-wide failure from a single injected error without intervention (arXiv 2603.04474). A minor drift in one agent becomes a serious problem two steps downstream.
Moving enforcement outside the model
The Agent enforces permission boundaries at the orchestration layer, separate from any individual model's inference. Every action an agent proposes passes through that layer before it executes. The orchestration layer evaluates the action across four dimensions. What the agent can access, what operations it can perform, which other agents it can invoke, and how long it can run.
The model does not decide whether an action is allowed. The orchestration layer does. No adversarial content reaching the model changes this, because enforcement happens in a separate system that processes no unstructured external input.
The principle is the same one that makes operating system permission systems work. Applications do not decide whether they can access a file. The kernel does, regardless of what instructions are inside the application.
The research question we are pursuing
We are actively measuring whether this architecture produces meaningfully better constraint satisfaction than prompt-based approaches in real multi-step autonomous agent workflows. Does runtime enforcement at the orchestration layer reduce violation rates, and what does it cost at scale?
This is not settled in the existing literature. Published work on runtime enforcement has demonstrated strong results in single-agent settings. The multi-agent, multi-provider case, which is what the Agent actually runs, has not been rigorously studied. We are building the system, measuring it, and will publish what we find.
Why this matters
Agents that operate autonomously are only useful if you can trust them to stay within bounds. Not because you hope they will, but because you can verify they do. The goal is AI that operates within boundaries enforced independent of its own inference.
For more on our research, go to gotmode.com/r-and-d. For more on our safety architecture, go to gotmode.com/safety.