Direct Answer: Treat AI Agents as Untrusted Execution Systems
Enterprises should control AI agents at execution time rather than relying only on model training, system prompts, or pre-deployment testing. An AI agent runtime security control point sits between the model’s intended request and the tools, data, credentials, and business systems it can actually reach. It evaluates each proposed action against permissions, user identity, task context, data classification, destination, and risk thresholds before allowing, modifying, pausing, or terminating execution.
Also worth reading: What is AI agent credential management and how should enterprises handle credentials for autonomous agents in 2026? · What Is Runtime AI Governance Architecture and How Should Multi-Team Enterprises Design It in 2026? · Command center vs operations dashboard for enterprises: what is the difference and which should leadership choose?
This approach is not equivalent to conventional application security, although runtime protection products may also cover cloud workloads, vulnerabilities, and network activity. The distinguishing problem is that an agent can interpret natural language, choose tools, generate code, call APIs, and revise its next step without following a fixed, prewritten transaction path. Static rules may approve a harmless-looking action while overlooking how that action becomes dangerous when combined with earlier steps. A runtime decision therefore needs context about the whole action chain, not merely the text of one prompt.
The direct answer for leadership teams running multi-team operations is to begin with agents that can send email, access customer records, change production infrastructure, move money, or expose confidential material. Purely conversational agents with no external tools create less direct risk, but they can still disclose training inputs, leak system instructions, or generate harmful content. By September 2026, the market is active enough to justify the category: Arrakis announced an $8 million round for AI agent runtime security, while Kontext Security emerged with $4 million for runtime controls. Those investments show buyer interest, not proof that any one vendor’s claims are correct.
A practical operating model is “decide, inspect, constrain, record, and respond.” Decide whether the action is permitted; inspect its arguments and context; constrain it through least privilege and transaction limits; record the model, prompt, policy decision, tool response, and human approvals; and respond when behavior deviates from expectations. This control point should be capable of stopping an action, not merely generating a warning after a breach. Research presented in 2025 and 2026 increasingly focuses on injection, tool abuse, data exfiltration, and the need to intervene before execution.
How AI Agent Runtime Security Actually Works
A runtime control point normally intercepts tool calls or agent-generated code. Before execution, a policy engine evaluates who initiated the request, which agent is acting, what data it is requesting, which system it will affect, whether the action is reversible, and whether the current sequence is consistent with the assigned task. For example, an agent allowed to search a customer database should not automatically receive permission to export every matching record to an external endpoint. A payment agent may be authorized to prepare an invoice but not to transmit it without approval above a defined threshold.
The system then applies controls at several levels. Identity controls bind the agent to a temporary, task-specific identity rather than a permanent human account. Data controls can mask secrets, restrict fields, limit row counts, or block files classified above a chosen level. Network controls restrict destinations and protocols. Tool controls constrain arguments, methods, and scope. Behavioral controls detect unusual sequences, such as repeated failed authentication followed by access to a sensitive repository. Human approval can be inserted for high-impact actions, while automatic termination can stop runaway loops or policy violations.
The word “runtime” matters because controls must account for what the agent does after deployment. Traditional secure software development reviews source code and known vulnerabilities before release, but an agent can construct novel action paths from ordinary tools. Security researchers have discussed threats including prompt injection, indirect instructions embedded in retrieved documents, excessive permissions, arbitrary code execution, and exfiltration. Hugging Face disclosed a cyberattack by autonomous AI agents in July 2026, according to the supplied research context, illustrating why AI-driven activity needs detection and containment rather than assumptions of benign intent.
No control can reconstruct intent perfectly. A model may misunderstand an ambiguous instruction, a policy engine may create false positives, and an external service may change after an action has been approved. Runtime security therefore combines deterministic authorization with behavioral monitoring, testing, incident response, and human judgment. The strongest systems make uncertainty explicit: they can route low-risk actions automatically, require review for sensitive actions, and fail closed when identity, context, or policy state cannot be verified.
Why Predeployment Testing Is Not Enough
Predeployment evaluation remains necessary because it can find weak prompts, unsafe model behavior, and tool configurations that should never reach production. It is also considerably easier and cheaper to correct a bad permission before an agent can act. Model red teaming can test whether an agent follows injected instructions in a web page, retrieves confidential files, or bypasses a stated restriction. Security reviews can identify standing credentials, broad API scopes, and missing approval gates.
However, predeployment testing cannot enumerate every runtime condition. Agents encounter changing documents, changing user requests, changing tool responses, and changing permissions. An apparently safe connector may expose millions of records to an agent that has been given no useful data-minimization requirement. A harmless command can become destructive when the agent has accumulated credentials from several earlier tools. A tool can also return content containing instructions that attempt to redirect the agent, creating an indirect prompt-injection path that was not present when the system was tested.
This does not mean runtime controls should be treated as a substitute for secure design. If an agent has unrestricted administrator credentials, strong runtime policy may be difficult to apply consistently and may generate too many alerts to investigate. Security teams should still use short-lived credentials, isolated execution environments, signed code where appropriate, dependency review, secure model gateways, and narrowly scoped APIs. Runtime protection then serves as a final decision layer across otherwise disciplined components.
The balance also depends on autonomy. A read-only assistant that summarizes public documents needs a different control profile from an agent that modifies customer accounts or deploys code. Testing should be continuous after every model update, prompt change, tool change, and permission change. As a practical trigger, teams can re-test whenever an agent gains access to a new data source, can send data outside the organization, can execute generated code, or can affect a system whose failure would create legal, financial, or customer harm. A runtime control point is most valuable when production can change faster than traditional release testing.
Practical Controls for Multi-Team Operations
Start by inventorying agents, owners, users, tools, identities, data sources, and external destinations. Record what each agent is supposed to do and which actions would be unacceptable. A control matrix should connect business purpose to permitted tools and maximum impact; otherwise, security teams may receive thousands of tool names without knowing which combinations matter. Assign one accountable owner to each production agent, even if several teams contribute prompts, integrations, and data.
Next, replace broad credentials with short-lived, task-scoped identities. Give an agent permission to read one approved repository rather than the entire cloud account, or permission to draft a refund rather than issue one. Separate planning identities from execution identities so that an agent cannot approve its own high-risk action. Remove default credentials from prompts and context windows, and store secrets outside the model’s reachable content whenever possible.
Then establish risk tiers and measurable gates. One workable internal policy might permit read-only internal queries automatically, require approval for external email or customer-record export, and block account deletion, privilege changes, or production deployment. These are starting thresholds, not industry-wide standards; a bank, hospital, and software company should choose different limits. Add limits for action count, data volume, spending, runtime duration, and repeated failures to prevent an agent from consuming resources or generating excessive business impact.
Finally, make every decision observable. Store timestamps, the initiating user, agent version, prompt or policy context, requested tool, arguments after sensitive-field masking, policy result, approver, and resulting action. Alert on policy denials, unusual destinations, abnormal volume, privilege escalation, and sequences that contradict the agent’s assigned role. The operations team needs a command view that shows what happened across teams, while security and system owners need enough evidence to investigate. A dashboard that only reports “blocked” without context is not an effective control point.
Comparison of Runtime Protection Approaches
Organizations can combine several approaches instead of searching for a single product category. The choice depends on whether the priority is fast deployment, code-level isolation, network protection, data governance, or complete activity visibility. The table below compares common approaches; it is not a vendor ranking and does not imply that one layer can replace the others.
| Feature | Agent-specific control point | Cloud and workload protection | Identity and access management | Human approval workflow |
|---|---|---|---|---|
| Primary purpose | Evaluate agent intent, context, tools, and action sequences | Protect applications, containers, networks, and cloud resources | Govern users, service accounts, roles, and secrets | Add human judgment before high-impact actions |
| Best for | Tool-using agents with dynamic behavior | Infrastructure and runtime workload threats | Credential scope and authorization | Irreversible or unusually sensitive operations |
| Typical visibility | Prompt, model, tool call, data request, destination, approval | Process, file, network, package, cloud activity | Identity, role, token, session, privilege | Request, reviewer, reason, decision, timestamp |
| Strength | Context-aware pre-execution decisions | Broad protection across many workload types | Durable identity boundaries and access governance | Clear accountability for consequential actions |
| Limitation | Policy quality and context accuracy determine results | May not understand an agent’s task or planned sequence | Usually does not inspect all model-generated content | Can be slow and may be bypassed if applied to the wrong action |
| Example control | Block an agent from exporting records to an unapproved endpoint | Terminate a suspicious process or block malicious traffic | Issue a 15-minute token for one repository | Require a finance manager to approve a $25,000 transfer |
Cost should be evaluated across prevention, investigation, recovery, and engineering time. Public funding figures such as Arrakis’s $8 million and Kontext Security’s $4 million are financing announcements, not customer prices. The supplied research does not establish a standard list price for agent runtime control, and vendors may price by agent, user, protected tool, workload, event volume, or enterprise contract. Compare total operating cost rather than assuming that an inexpensive connector is cheaper once incident response, credential rotation, and compliance evidence are included.
Alternatives, Open-Source Projects, and Buying Criteria
Organizations have several alternatives. They can use conventional API gateways, service meshes, web application firewalls, data loss prevention tools, sandboxing platforms, and identity governance products. These can constrain networks, schemas, secrets, or access, but they may not understand the semantic purpose of an agent’s action. An API gateway can enforce a field-level schema; it may not know that an agent is using a valid read endpoint to assemble an unauthorized export.
The research context also points to open-source and developer-oriented efforts, including the Agent Governance Toolkit, ButterClaw, Burrow, and projects described as runtime security for AI agents. Open-source tools can provide useful policy models, sandbox boundaries, or integrations, especially for technical teams willing to operate them. They do not automatically provide enterprise support, reliable updates, complete audit functions, or a tested response process. A project that terminates a process on suspected breach should also be evaluated for false positives, denial-of-service exposure, telemetry handling, and operational recovery.
A procurement process should ask how a product makes decisions. Can administrators express controls by identity, tool, data classification, destination, action cost, and approval level? Does it inspect every tool call or only network traffic? Can it revoke credentials and terminate an active run? Does it support local, private-cloud, or air-gapped deployments where required? How are prompt-injection and data-exfiltration alerts explained to an investigator? Can policies be tested in a simulation before production enforcement?
Teams should also ask what happens when the policy service is unavailable. A fail-closed design may protect sensitive systems but can stop legitimate operations; a fail-open design may preserve availability but expose data. A staged response can deny only high-risk actions while allowing explicitly classified low-risk work, with a visible emergency override. The right choice depends on business impact, not on a universal security slogan.
Vendor claims require independent testing. Use representative tasks, adversarial documents, malformed tool output, attempted privilege escalation, and large data exports. Measure blocked attacks, false positives, decision latency, recovery time, and the percentage of actions with complete audit records. The category is developing quickly, so a short proof of concept is not enough; test model updates, new tools, and changed business rules over several weeks.
Common Mistakes and Failure Modes
The first mistake is assuming that a strong system prompt is a security boundary. Models can be influenced by direct or indirect instructions, and a prompt is not a reliable substitute for authorization. The second is giving an agent a broad human identity. If the agent uses a shared administrator account, the runtime cannot reliably distinguish legitimate actions from misuse, and incident response becomes harder because every event appears to come from the same principal.
Another common error is treating all anomalies as malicious. Agents may retry failed calls, consult several sources, or generate longer actions than expected. A control system that blocks every unexpected sequence can train teams to disable it. Teams should distinguish confirmed policy violations from weak signals such as a new destination, a new tool, a high-volume query, or a repeated retry. Baseline normal behavior, define investigation thresholds, and measure precision as carefully as detection coverage.
Teams also underestimate data movement. A tool can be technically internal while sending data to a contractor, an analytics service, a model provider, or a personal account. Conversely, an external call may be safe if no sensitive data is included. Runtime controls should evaluate both identity and data, including redaction, tokenization, file classification, and destination reputation. Logging the full prompt can itself create a new exposure if logs contain secrets or regulated information.
Finally, many programs lack an owner for the final decision. Security may define policy, but the business team must decide acceptable impact and approve exceptions. A useful governance forum reviews new agents, high-risk actions, denied requests, near misses, and changes in autonomy at least monthly during a rollout. The program should define a kill switch, a credential-revocation procedure, and a tested path to continue essential operations without the agent. Without those practices, “runtime security” becomes an alert feed that nobody can act on during an incident.
When to Act and How to Measure the Program
Act now when an agent can access sensitive information, use credentials, execute generated code, contact external systems, or make decisions that affect customers or revenue. The risk is not determined only by model size or traffic volume; a small model with administrator access can be more dangerous than a large model used for public content. Organizations should also act when autonomy is increasing, when several teams share tools, or when an agent’s behavior can change after deployment.
A staged rollout can start in observation mode for 2 to 4 weeks, followed by enforcement on the highest-risk actions. During observation, record proposed calls, data access, destinations, and unusual sequences without changing behavior. Compare the results with the agent’s stated purpose and identify missing policy. Then enforce deny rules for known unacceptable actions, add approval requirements for consequential actions, and retain automatic access only for low-risk operations that have been tested.
Use metrics that connect security activity to business operations. Track the percentage of agent actions with an identified owner, the number of standing credentials, the median time to revoke access, the percentage of tool calls covered by policy, the number of high-risk actions approved, denied, and investigated, and the time needed to contain a runaway run. Measure false-positive rates by team and action type, not only overall averages. A target of zero alerts is usually less useful than a target that every high-risk action is either prevented, approved with a named reviewer, or investigated within an agreed service level.
The program should be reviewed after material changes. A reasonable trigger is any new model version, tool, data source, external destination, or permission scope, as well as any incident involving an agent. If the agent begins handling a new regulated or customer-impacting workflow, repeat the threat model and approval design. This is continuous governance because the relevant behavior comes from the combination of model, prompt, context, tools, permissions, and external systems—not from the model alone.
A Recommended Operating Standard for Leadership Teams
For a B2B command-center SaaS environment coordinating multiple teams, the practical standard is that every production agent has a named business owner, a bounded identity, an inventory of reachable systems, and a documented action policy. Leadership should receive a concise view of which agents are active, what they can do, what they have done, and which risks are awaiting a decision. The view should distinguish policy denials from operational failures, because the same blocked action may indicate an attack, a broken integration, or an incorrectly configured permission.
Set three initial service tiers: low-risk internal analysis, sensitive-data or external-communication actions, and irreversible business changes. Permit the first tier through tested controls, require review for the second, and require explicit authorization for the third. Use transaction and volume thresholds as a starting point, then adjust them after measuring real workflows. For example, a team might cap an agent at 1,000 retrieved records per task, 15 minutes of execution, or 10 external messages before review, but those numbers should be based on data sensitivity and capacity rather than copied from a generic article.
Maintain evidence that the control is effective. Security and operations should jointly test prompt injection, indirect instructions in retrieved content, credential misuse, data exfiltration, and repeated tool failures. They should verify that a blocked action cannot continue, that credentials can be revoked, and that an operator can identify the exact agent run and business request. Quarterly tabletop exercises are sensible for agents with production or customer impact; more frequent testing is appropriate during major model or workflow changes.
The defensible conclusion is not that every organization needs an elaborate platform. Small deployments can begin with tightly scoped tools, short-lived credentials, gateway rules, and human review. The conclusion is that autonomy and execution authority must be managed together. By September 2026, the relevant question for leadership is no longer merely whether an agent produces a good answer, but whether the organization can prove what it was allowed to do, what it attempted, what it disclosed, and who remained accountable when the action was wrong.