Human approval is a system design decision, not a concession.

Consider an AI workflow that handles customer order adjustments. It reads a request, checks account details and proposes a change. The system can save several minutes per transaction if it prepares a recommendation. It can also create a serious problem if it commits the wrong adjustment without the required authorization.

Should the organization give the agent autonomy? That question is too broad. The better question is which actions are safe under which conditions, and who is accountable when the conditions are not met.

01. Separate interpretation from execution

Language models are useful where input is messy, a document requires synthesis or a user needs help understanding possible options. Business systems need deterministic state changes. An approval can be a deliberate boundary between those two functions.

An agent may extract candidate fields, prepare a draft, compare a policy or suggest next steps. A rule engine should validate ordinary constraints, and a user with the right role should approve high-consequence actions. The eventual API call should come from a controlled service, not an unrestricted model tool.

This separation makes it easier to test what the system does and to reason about its failure modes.

02. Approval belongs before a consequential transition

A workflow state machine might include Draft, Evidence Ready, Validation Failed, Pending Approval, Approved, Submitted and Reconciled. The state transitions should be meaningful business events. A prompt that says “ask for confirmation” is not the same as an authorization check in the backend.

The approval record should contain the relevant evidence, what the system proposed, what the human confirmed, the actor's identity, time and the version of the rule that allowed the change. If the downstream API fails, the system needs a recovery state. A user should not be told an order was updated merely because an AI message said it was.

03. The adversary may arrive through a document

An agent that reads an external email or attachment may encounter text instructing it to ignore policies or take unrelated actions. That is untrusted content, not a privileged system instruction. Prompt injection is a practical threat, particularly when an application gives a model tools with access to valuable data or write permissions.

OWASP's guidance on LLM applications identifies excessive agency as a risk when an AI system has too much functionality, permission or autonomy. The remedy is not simply to write a stronger prompt. It is to restrict tools, validate actions and preserve human or deterministic control. [1]

04. Repeated actions are a business problem

Suppose the service submits a change but times out before receiving confirmation. It retries. If the endpoint is not idempotent, the action may happen twice. This is a conventional distributed systems problem, now made more likely to be overlooked by systems marketed as agentic. The design needs transaction identifiers, reconciliation and a way to determine what actually happened.

A dependable AI workflow has at least three kinds of test: the model chooses a reasonable interpretation, the policy system permits only approved actions, and the integration updates the authoritative record exactly once or recovers safely.

05. Increase autonomy only with evidence

For low-risk routine tasks, human review may be unnecessary after representative testing. For decisions involving money, customer rights, safety or confidential information, additional controls can be appropriate. There is no universal autonomy level. The test is consequence, reversibility, data and demonstrated failure rate.

The most useful metric may not be tasks completed without humans. It may be correct tasks completed, time to resolve exceptions, and how quickly the organization can detect and recover from an error.

When review can become a bottleneck

Poorly designed human approval can merely move the work to another queue. The review interface should surface what matters: the recommendation, supporting evidence, policy result and the specific change awaiting confirmation. The goal is not to make every employee read the full document again.

The practical rule: Automate interpretation where it helps. Guard consequential transitions with deterministic checks and proportionate human control. Treat the exception path as a first-class product experience.

Sources and limitations

[1] OWASP, Top 10 for LLM Applications, 2025.
[2] NIST, Generative AI Profile, 2024, https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence
[3] Thoughtworks Technology Radar Volume 35, November 2026, https://www.thoughtworks.com/radar