Orchestration-layer enforcement
vs. prompt-based approaches.
Mode is investigating whether runtime enforcement of permission boundaries at the agent orchestration layer provides more robust constraint satisfaction than model-layer prompt-based approaches in autonomous multi-step AI workflows.
The research question
Does enforcing permission boundaries at the agent orchestration layer, independent of any individual model's inference, reduce constraint violations more effectively than prompt-based approaches in multi-step autonomous AI agent workflows, particularly under adversarial prompt injection conditions, and what is the enforcement cost at scale?
This question sits at the intersection of AI safety, multi-agent systems, and systems security. The answer has direct implications for how organizations deploy autonomous AI agents in production environments where unintended behavior carries real consequences.
Why existing approaches fall short
Most AI safety mechanisms today rely on instructing the model to behave safely. This approach has three well-documented failure modes.
Prompt injection bypasses prompt-based constraints
When an AI agent processes external content such as web pages, documents, or tool outputs, adversarial instructions embedded in that content can override the agent's original safety instructions. Research found that just five carefully crafted documents among millions can achieve 90% attack success through content poisoning, and that fool-proof prevention methods remain unclear due to the stochastic nature of LLMs. Safety instructions that exist only inside the model's context window are vulnerable to anything that can modify that context.
Errors cascade in multi-agent workflows
In multi-step agent workflows, a single constraint violation does not stay isolated. It becomes input to downstream agents, compounding through the task graph. Intent-hiding attacks that frame malicious content as credible achieve 85 to 100 percent success in tested multi-agent systems. Five of six tested agent frameworks reach 100% system-wide failure from a single injected error without intervention, with hub-node injection spreading at up to 10.3 times the rate of isolated errors. Production multi-agent systems fail at rates between 41% and 86.7% across tested frameworks, with system design and specification failures accounting for approximately 44% of recorded failure modes.
Context growth degrades instruction compliance over time
Even without adversarial input, safety instructions that live in the context window face a natural operational failure mode. As sessions extend across many tool calls and agent turns, context fills. Most frameworks handle this by compressing earlier content into summaries, truncating oldest turns, or using sliding windows that drop content past a length threshold. Safety instructions present at session start may be summarized into shorter representations, displaced toward the middle of a long context where retrieval performance can degrade significantly, or removed entirely. No attack is required. The model may simply no longer have reliable access to the constraints it was given. Orchestration-layer enforcement evaluates every action against a fixed configuration outside the model's context, making it immune to this degradation regardless of session length.
Mode's approach
Mode Agent enforces permission boundaries at the orchestration layer rather than through instructions passed to individual models.
Enforcement outside the model
Constraints are rules evaluated by the orchestration layer before any action executes. A model cannot override them through its outputs because the enforcement mechanism is independent of the model's inference. Prompt injection attacks that modify a model's context window cannot alter orchestration-layer constraints.
Heterogeneous model coverage
The Agent routes tasks across 600+ models from 100+ providers. A single enforcement layer governs all of them regardless of the individual model's instruction-following behavior or safety training. Enforcement does not depend on any one model's compliance.
Four-dimensional permission system
Boundaries are defined across scope (what the agent can access), operations (what actions it can take), agents (which other agents it can invoke), and duration (how long it can operate). Every executed action is evaluated against all four dimensions before execution proceeds.
Complete audit trail
Every agent action is logged at the orchestration layer, independent of what the model reports. The log captures what was attempted, what was permitted, and what was blocked, providing forensic visibility across the full task execution graph.
The gap in existing research
Recent published work on runtime enforcement for AI agents, including AgentSpec, Agent Behavioral Contracts, and ProbGuard, addresses enforcement at the single-agent level against a single model. None of this work addresses enforcement at the orchestration layer governing multiple agents routing across heterogeneous model providers.
The multi-agent, multi-provider setting introduces failure modes that single-agent benchmarks cannot measure. When enforcement must hold across models with different instruction-following characteristics, and when agents route tasks to providers dynamically, the question of whether orchestration-layer enforcement provides stronger guarantees becomes both practically important and scientifically open.
What we are measuring
Our evaluation framework compares three conditions across the same task set: no enforcement, prompt-based enforcement, and Mode Agent's orchestration-layer runtime enforcement.
Primary metrics, directly answer the research question.
Constraint violation rate
What percentage of agent actions breach defined permission boundaries under normal operation across all three enforcement conditions.
Prompt injection resistance
Under adversarial input including direct and indirect injection, what percentage of attempts successfully override intended constraints.
Behavioral drift
Over extended multi-turn sessions, does violation rate increase from start to end, indicating degradation of constraint compliance over time.
Secondary metrics, quantify the enforcement tradeoff. Published research on prompt injection defenses established that no single approach can simultaneously achieve high trustworthiness, high utility, and low latency. Measuring these dimensions for orchestration-layer enforcement in multi-agent settings is the empirical contribution this work makes.
Enforcement latency
Milliseconds of overhead added per agent action by runtime enforcement compared to no enforcement.
False positive rate
What percentage of legitimate, user-intended actions enforcement incorrectly blocks.
Audit completeness
What percentage of agent actions appear in the audit log, including attempted violations.
Related work
The research below establishes the problem space, motivates our approach, and defines the gap our work addresses.
AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents
Introduces a domain-specific language for runtime constraint enforcement on LLM agents. Achieves over 90% prevention of unsafe executions in code agent settings with millisecond overhead. Addresses single-agent enforcement against a single model.
arxiv.org/abs/2503.18666. March 2025Agent Behavioral Contracts: Formal Specification and Runtime Enforcement for Reliable Autonomous AI Agents
Brings design-by-contract principles to autonomous agents with formal preconditions, invariants, and recovery mechanisms. Introduces AgentContract-Bench across 200 scenarios, 7 models, and 6 vendors. Contracted agents detect 5.2 to 6.8 soft violations per session that uncontracted baselines miss entirely.
arxiv.org/abs/2602.22302. February 2026ProbGuard: Probabilistic Runtime Monitoring for LLM Agent Safety
Extends runtime enforcement to proactive violation prediction via probabilistic model checking. Anticipates violations before they occur by modeling agent behavior as symbolic abstractions through discrete-time Markov chains.
arxiv.org/abs/2508.00500. 2025From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration
Demonstrates that five of six tested agent frameworks reach 100% system-wide failure from a single injected error without intervention. Hub-node injection spreads at up to 10.3 times the rate of isolated errors. Identifies cascade amplification, topological sensitivity, and consensus inertia as the three primary vulnerability classes in multi-agent systems.
arxiv.org/abs/2603.04474. March 2026Why Do Multi-Agent LLM Systems Fail?
Empirical study of failure modes in production multi-agent systems. Empirical analysis reveals 41% to 86.7% failure rates across 7 tested multi-agent frameworks. System design and specification failures are the largest failure category, accounting for approximately 44% of recorded failures across 14 identified failure modes.
arxiv.org/abs/2503.13657. March 2025Prompt Injection Attacks in Large Language Models and AI Agent Systems
Comprehensive review of prompt injection vulnerability across LLM applications. Finds that five carefully crafted documents among millions achieve 90% attack success through content poisoning, and that fool-proof prevention methods remain unclear due to the stochastic nature of LLMs. Establishes that LLMs cannot reliably separate instructions from data at the model layer.
MDPI Information, 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysis
Systematic review establishing a taxonomy of prompt injection attacks by payload generation strategy and defenses by intervention stage (text, model, and execution levels). Introduces the AgentPI benchmark for context-dependent agent evaluation, finding that many defenses appear effective on existing benchmarks but fail to generalize to realistic agent settings. Core finding: no single approach can simultaneously achieve high trustworthiness, high utility, and low latency.
arxiv.org/abs/2602.10453. February 2026Agent Contracts: A Formal Framework for Resource-Bounded Autonomous AI Systems
Extends the Contract Net Protocol with formal resource governance, unifying input/output specifications, resource constraints, temporal boundaries, and success criteria into a coherent governance framework for bounded autonomous AI deployment.
arxiv.org/abs/2601.08815. January 2026