Engineering autonomy is a control design problem.

An employee asks an assistant to resolve an invoice mismatch. The assistant gathers purchase order context, identifies a likely discrepancy and suggests a credit. On a prototype screen, a friendly 'Apply change' button makes the experience feel finished. The difficult part is invisible: whose credentials authorize the action, what exactly will be written, and what happens if the API times out after it succeeds?

Autonomy is not simply a model choosing among tools. It is an application allowing a non-deterministic component to request actions inside a deterministic business system.

Start with authority, not tool selection

For every proposed action, name the identity under which it will execute, the resources it may read, the records it may change, and the conditions under which it may do so. The model should not gain broader permissions because a user puts persuasive instructions in a document or a chat message.

A read-only retrieval operation may be safe for wider delegation if source permissions are respected. Creating a draft ticket is different from sending a customer email. Updating a quote is different from issuing a refund. The design needs those distinctions before deciding whether the agent will have a single tool, a planning loop or a multi-agent orchestration system.

Give the workflow explicit states

A good state machine might contain: received, evidence gathered, proposed, review required, approved, commit pending, committed, reconciliation pending, failed and closed. Each transition has an owner and a rule.

The UI should show the state as clearly as the model's reasoning. A reviewer must know which action is merely proposed, what will change if approved, what evidence supports the change and whether the approval has already been used. Final policy enforcement belongs in the application, not the natural-language prompt.

Network failures are an autonomy problem

Suppose the destination API successfully creates a credit, but the response is lost. An agent that sees a timeout may repeat the request and create a second credit. This is not a hallucination. It is a distributed-systems failure.

Use idempotency keys where supported, transaction state, correlation IDs, compensation patterns or reconciliation queries as appropriate. Retry rules should be documented per action. Sensitive updates may need a human review checkpoint before the first commit and an explicit operator escalation before retrying uncertain outcomes.

The system also needs rate limits, maximum spend or action bounds, exception queues and alerts for repeated failures. A model's confidence in its own plan cannot override these controls.

Test the malicious instruction, not just the happy path

Place a simulated prompt injection in an incoming email instructing the agent to send payment data to a different account. The correct system behavior is not merely to answer that it should ignore the instruction. It is to prevent any unauthorized destination from being passed to the write API.

Test a user whose permission was revoked midway through a workflow. Test two pending approvals on the same item. Test a failed external API and a model provider outage. Test a human who corrects the proposed value. Test access across organizations or tenants when the system is multi-tenant.

A useful autonomy ladder

The same business workflow can support different degrees of delegation. Begin with read-and-suggest. Then drafts that the user reviews. Then bounded actions with strong deterministic validation, reversible changes and recorded approvals. Allow higher autonomy only if repeated evaluations, operational monitoring and consequences justify it.

There is no universal safe level for every organization. A low-stakes draft meeting summary and an irreversible financial action have different risk profiles.

The decision: Treat the agent as a proposer with bounded authority. Let architecture enforce what the business has approved.

References and limits

OWASP Top 10 for LLM Applications; NIST Generative AI Profile. Technical recommendations vary by system risk; not a claim of certified safety.