# Autonomy should be earned, not assumed.

> An AI agent can suggest a next step or use a connected tool. The harder question is what it is allowed to do, when it must ask a person and how it recovers from a mistake.

Canonical URL: https://marmation.com/expertise/ai-agent-orchestration/

## The decision

What checks must pass before an assistant can change a customer record?

Define which steps may read, propose, approve and save. Enforce those rules in application code and tool permissions; model confidence or instructions in an attachment cannot authorize a change.

## A service assistant can reach the wrong customer account

An assistant can open a service ticket, look up account data and propose an adjustment. An untrusted attachment could contain instructions to access unrelated accounts or change the amount.

## Instructions in a document cannot grant permission

Instructions telling the model what to do cannot replace enforced access checks. If tools can reach unrelated accounts, nobody can see which workflow step is active, or a retry can repeat a business action, a useful assistant can cause an unauthorized change or duplicate adjustment.

## Separate reading, proposing, approving and saving

Use an explicit workflow or state machine: a record of the current step and the conditions for moving to the next one. Give the model narrow tools and read-only access by default. Separate proposing a change from approving and saving it. Preserve request identity across retries, check uncertain results at the destination and define when a person must resolve a conflict or authorize recovery.

## Controls for tools, approvals and recovery

- Tool and identity permissions matrix.
- State machine, action gates and human-review interface.
- Retries that cannot duplicate an action (idempotency), reversing transactions where supported (compensation), and an audit history.
- Prompt-injection, data leakage and failure evaluation.


## One classification step may be enough

The right design may use zero autonomous agents and one classification step. Multi-agent collaboration is a means, not the objective.

## Try duplicate actions, hostile inputs and failed tools

Test repeated actions, partial failure, malicious tool outputs, conflicting instructions, false confidence, authorization bypass and rollback feasibility.

## Define what the assistant must never do

Take one bounded task and define what the agent may not do. A permission list is more useful than a demo video.

## Related reading

- [Give an agent a job. Give the system boundaries.](https://marmation.com/solutions/ai-agents-and-workflows/)
- [Threat-model the tools, not just the prompts.](https://marmation.com/review/threat-modeling-tool-using-ai-agents/)
- [Governance works best when it is part of the system design.](https://marmation.com/expertise/ai-data-and-model-governance/)