Autonomy is not a slider
The common way to talk about agent autonomy is as a dial that gets turned up as confidence grows. It is a misleading picture. An application does not have one level of autonomy; it has a different level for every capability it holds, and those levels should have very little to do with how capable the model is.
What they should follow is consequence and reversibility. Reading a customer record is cheap and undoable. Sending that customer a letter about an unpaid invoice is neither. The same application can reasonably do the first thousands of times a day without supervision and never do the second without a named person approving it, and that is not a limitation to be engineered away. It is the design.
This matters commercially as well as technically. The failure mode that kills these projects is not a wrong answer. It is one embarrassing action taken without permission, after which nobody in the business will authorise anything again.
The four questions that set authority
Before granting an application any capability, four questions settle how much supervision it needs. They are worth asking per action rather than per system.
- What does it cost if this is wrong? Money, a relationship, a legal position, or nothing much.
- Can it be undone, and by whom? An unallocated payment is recoverable in minutes. A message to a customer is not recoverable at all.
- Who would notice, and how quickly? Silent failure deserves more supervision than loud failure, because it accumulates.
- Is there evidence it works? Not an impression from a demonstration, but measured performance on real cases with a known error rate.
What stays in ordinary code, permanently
Some things should never be produced by inference, regardless of how reliable the model becomes, because their correctness has to be provable rather than probable.
Money is the clearest case. Invoice totals, tax, credits, balances and the arithmetic of a payment plan belong in application code and the accounting system, where they can be unit tested and reconciled. An agent should quote a balance, never compute one. The moment a figure originates in a model, you have lost the ability to audit it, and the first time a customer disputes an amount you will not be able to explain where it came from.
Entitlement is the second. What a customer is owed, what their notice period is, whether they are within a warranty, whether a discount applies: these are contractual facts. They live in terms and in code, and the application reads them.
Permission is the third, and the one most often got wrong. Who may see which records is enforced by the application before context is assembled, not by asking the model nicely to respect a scope. If the only thing preventing a user from seeing another site data is an instruction in a prompt, there is no access control.
The fourth is the audit record. What was read, what was proposed, who approved it and what was executed has to be written by the system as a side effect of doing the work, not summarised afterwards by the thing being audited.
Actions that should stay behind approval
Beyond the things that should never be inferred, there is a set of actions that an agentic application may well prepare, and should not perform unsupervised in any early release. Some of them should stay supervised permanently.
- Anything that leaves the building. External messages, particularly about money, complaints or anything a customer might forward to a solicitor.
- Anything that moves money. Charging a card, issuing a refund, allocating an unmatched payment, writing off a balance.
- Anything that changes a contract. Cancellations, renewals, variations, terminations and access restrictions.
- Anything irreversible. Deletions, dispatches, published content and actions that trigger third-party processes.
- Anything that escalates. Legal steps, formal notices, account suspension and complaint responses.
Approval has to be workable, or it will be bypassed
A control that costs more attention than the work it supervises will be defeated within a month, usually by someone approving a batch without reading it. That is worse than no control, because it manufactures the appearance of oversight.
Good approval is batched, evidenced and fast. A person sees twenty prepared items in one screen, each with the reason it exists and a link to the records behind it, edits the two that need editing and approves the rest in a single pass. The measure of whether this is working is the edit rate: heavy editing means the application is being given the wrong context, and that is a defect to fix rather than a cost to absorb.
The other half is what the application refuses to send. In practice a well-designed collections batch contains items that are not chases at all: an account on an agreed plan that should be left alone, a short payment that is a billing error rather than a debt, an overdue invoice attached to an unresolved complaint where escalating would be a mistake. An application that surfaces those is doing the supervisor job properly. One that quietly drops them is hiding its uncertainty.
Treat every input as untrusted
An agentic application reads text it did not write: customer emails, uploaded documents, third-party API responses, notes typed by staff. All of it is data, and none of it is instruction. An application that will follow a sentence found inside an email because the sentence sounds authoritative has a vulnerability, not a feature.
The defence is architectural rather than linguistic. Capabilities are authorised by the application against the identity of the requesting user, independently of anything the content says. Scope is applied before assembly. A message claiming to be from a director does not become one. Tests should include deliberate injection attempts, and they should fail.
This is also why broad credentials are the wrong shape. A capability that can read one customer records cannot be talked into reading another. A service account with database access can be talked into a great deal.
Earning authority, one capability at a time
Authority should be granted the way you would grant it to a new colleague: narrowly, on evidence, for one thing at a time. The sequence that works is observation, then drafting, then reversible action, then, for a small number of low-consequence operations, unsupervised execution within explicit limits.
Each step needs its own evidence: a dataset of real cases, a measured error rate, an agreed threshold and a review of the errors that remain. "It seemed fine in testing" is not evidence, and neither is a low complaint rate over two weeks.
Limits should be enforced in code and should be boring: volume caps, spending caps, rate limits, time windows. A defect should be capable of preparing a hundred wrong drafts and structurally incapable of sending them. That single property has saved more projects than any amount of prompt engineering.
The test worth applying
Before any capability goes live, ask what the worst realistic version of this action looks like on a bad day, with a stale record, a partial outage and an unusual customer. Then ask who finds out, how long it takes, and what undoes it.
If the honest answers are uncomfortable, the capability is not ready for autonomy, however well it performed in review. That is not caution for its own sake. Businesses adopt this kind of software on trust, and trust is spent much faster than it is earned.