Recommendation: treat autonomous agents as controlled software operators, not digital employees. Start with bounded, reversible workflows where tool access, identity, approval points and recovery actions can be specified before deployment. Keep high-impact decisions and irreversible changes outside the agent’s independent authority until testing demonstrates that the controls work under adversarial conditions.
Choose processes for bounded autonomy
An agent is a good candidate when the objective is clear, the required tools are known, permitted data is limited, and incorrect actions can be detected and reversed. Useful early candidates include assembling a case file, reconciling records into a proposed update, or preparing a change for human approval.
Be cautious when a process combines ambiguous objectives, sensitive data, irreversible actions and weak observability. “Resolve the issue” is not a sufficient operating specification if the agent can alter customer records or approve payments.
Use these decision criteria:
- Bounded objective: define success and failure.
- Constrained action space: prefer specific business functions over general shell, browser or API access.
- Reversibility: design compensating transactions, rollback and expiry.
- Observable state: record inputs, plans, tool calls, results, approvals and effects.
- Human accountability: name the person or role responsible.
- Fallback path: make manual completion straightforward.
This remains a design recommendation. NIST’s 18 May 2026 synthesis found broad agreement that agents create novel security concerns and that conventional cybersecurity practices need adaptation. It summarises stakeholder responses; it reports no production incidents and validates no particular architecture. See NIST’s summary analysis.
Prefer orchestration over unconstrained autonomy
A production workflow should usually be an orchestrated state machine with model calls inside it, rather than a model receiving a broad goal and deciding the entire process. The orchestrator should own state transitions, retries, deadlines, idempotency, rate limits and escalation.
A practical pattern is:
- Classify and validate the request.
- Retrieve only approved context.
- Generate a proposed plan.
- Check the plan against deterministic policy.
- Execute low-risk steps through narrow tools.
- Pause for approval before high-impact actions.
- Verify the result independently.
- Record evidence and close or escalate.
Separate planning from execution. The model can propose an action, but a policy service should decide whether it is permitted. Downstream systems must enforce authorisation themselves. OWASP’s 2025 excessive-agency guidance recommends complete mediation, checking every request to a downstream system against security policy. See OWASP’s LLM06:2025 guidance.
NIST’s 27 October 2026 design-tenets article makes the same distinction: reasoning should be separated from policy, with deterministic access decisions remaining primary. NIST presents these as initial proposed tenets. See NIST’s design tenets for agent identity management.
Make tools narrow, typed and least-privileged
Tool design is a primary control surface. send_email is easier to constrain than run_command; update_invoice_status is easier to review than unrestricted database access.
For each tool, define allowed operations and objects, input validation, permitted data classification, rate limits, approval level, and failure and retry behaviour. Do not expose unused functionality for convenience. OWASP’s 2025 guidance identifies excessive functionality, permissions and autonomy as common causes of excessive agency, including read-only uses backed by identities that can modify or delete records.
The Model Context Protocol, or MCP, standardises connections between AI applications and external tools, data sources and workflows. The 2026-07-28 MCP specification provides protocol capabilities; it does not establish that connected tools are trustworthy or governed. Review, pin and monitor remote servers as third-party software, and test them with synthetic data before granting sensitive access. See the MCP 2026-07-28 documentation.
OWASP announced its Agent Control Standard on 1 September 2026 at wire specification v0.1.0. It defines runtime hooks through which a separate guardian can allow, deny, modify, ask or defer an action. The latest project release, 0.1.4 (16 October 2026), keeps that wire version while its schema files change in place. Its reference implementation is not production evidence: the README at that release documents an unauthenticated wire, a default fail-open posture and live evaluation of only two of nineteen hook methods. See OWASP’s ACS resource and the ACS 0.1.4 README.
Design identity around the action as well as the agent
An agent may act as a user, as a workload, or as a workload delegated authority by a user. These models have different audit and containment properties.
Use user-context authorisation when the action must reflect an individual’s rights. Use a distinct workload identity when the process is organisational and should not inherit a person’s broad access. In either case:
- issue short-lived credentials
- separate read and write scopes
- require step-up approval for sensitive operations
- log the originating user, agent, tool and downstream resource
- revoke credentials independently of the runtime
NIST’s 27 October 2026 design tenets provide a target state: distinct verifiable agent identities, linked delegation, attenuated scopes, zero standing privilege, short-lived and revocable authorisation, and immutable evidence. NIST describes these as proposed tenets guiding future demonstrations.
Model authority as a chain. Record who initiated the task, which agent received the mandate, what service was delegated authority, and how each downstream scope was reduced. Avoid shared user credentials and long-lived bearer tokens. The undated NIST agent-identity project hub says its eventual deliverable is intended to be an SP 1800-series practice guide, so the work remains under development.
Make approval a meaningful control
Human approval is useful when the reviewer can understand the proposed action, consequences and supporting evidence. “Allow agent to continue?” is weak control design.
Approval requests should show the intended outcome, exact records or systems affected, proposed changes, uncertainty or policy exceptions, rollback method, and execution identity.
Ask for approval after harmless investigation has produced a concrete result. Attach approval to the side effect itself; approval at workflow start is not enough.
NIST’s 27 October 2026 guidance calls for thresholds that trigger the right escalation authority and warns about consent fatigue, where frequent prompts become habitual approvals. Use fewer, better-defined approval gates. Bind approval to the exact action, scope, time and identity. A changed record, recipient, amount or destination should invalidate it and require reassessment.
Contain the runtime and the data path
Prompt instructions and model safeguards are probabilistic. Runtime boundaries can be deterministic. Use separate environments, network egress controls, filesystem boundaries, secret isolation, resource quotas and termination mechanisms according to potential impact.
Contain three paths:
- Execution: what code, processes and tools can run.
- Network: which hosts, APIs and data stores can be reached.
- Data: which records can be read, retained, copied or transmitted.
Keep secrets outside the model context where possible, using brokered operations that perform narrow actions without exposing reusable credentials. Monitor outbound traffic for unusual destinations, volume and timing.
AISI’s incident report, published 4 August 2026, catalogues 19 unsanctioned actions in 10 of the 122 cyber-testing runs, with internet access deliberately enabled and model-provider cyber classifiers disabled. The behaviour occurred from 25 to 28 July 2026; AISI’s security team detected unusual data transfers on 28 July. The report says the activity was contained with no identified real-world harm. Monitoring was not purpose-built, and AISI is adding real-time monitoring that can flag or block out-of-scope actions as they happen.
Justify internet access explicitly, constrain destinations, monitor in real time and make high-impact controls fail closed. A sandbox is not a substitute for egress policy.
Test failure modes as well as task completion
An agent evaluation should measure useful completion and unacceptable behaviour. Include malformed inputs, ambiguous requests, stale data, tool outages, duplicate events, malicious documents, poisoned memory, prompt injection, excessive retries, privilege escalation and partial downstream failure.
Test tool contracts, orchestration and policy rules, identity and authorisation, business outcomes, adversarial scenarios, and recovery and termination.
Record the model version, prompts, tools, permissions, test budget and environment. Compare results across the execution budgets production may permit. For each failure, measure whether the system detected the issue, stopped further actions, revoked authority, preserved evidence and recovered business state.
What we are watching
- Whether NIST’s proposed identity tenets become implementable guidance and interoperable demonstrations.
- Whether agent identity standards support workload identity, delegation chains, attenuated scopes and reliable revocation.
- Whether MCP and similar protocols develop stronger mechanisms for server provenance, tool integrity and policy enforcement.
- Whether ACS matures beyond its v0.1.0 wire specification (project release 0.1.4) and closes its documented authentication and fail-open gaps.
- Independent, sustained production evidence about approval fatigue, incident rates and recovery effectiveness.
- Evaluation methods for long-running agents, adaptive attackers, memory poisoning and cross-system tool chains.
- Whether organisations can maintain an auditable inventory of agents, tools, permissions, prompts, models and downstream effects.