Should your startup let an AI agent act? Define the authority envelope first

Let an AI agent act only after you can state, in plain language, what it may read, what it may change, which actions need approval, how much cumulative damage one run can cause, what evidence every action leaves, and how a human stops or reverses it. If any field is blank, keep the system in read-only or recommendation mode.

The important AI release pattern this month is not a smarter chat box. Agents are moving into email, purchases, refunds, and other state-changing workflows. That makes the founder's architecture decision less about model intelligence and more about bounded authority.

What changed with the latest agent launches?

Meta's September 8 Muse announcement says its new personal agent can work after the app closes, use connected applications, fill forms, and return for approval before sending an email or making a purchase. Meta also says Muse isolates each agent, separates credentials from the model, lets people choose application permissions, and exposes an audit trail. Those are Meta's product claims, not independent proof of the controls, but the design choices reveal where the hard problem has moved.

Google's September 15 zero-trust agent guide makes the same shift explicit for a refund workflow. It describes screening prompts, evaluating a proposed tool call against user intent and business policy before execution, and monitoring cumulative behavior across a session. A single refund can be valid while a sequence of individually valid refunds drains the ledger.

The lesson for a startup is not “copy Meta's architecture” or “buy Google's platform.” It is this: a model can produce the right tool syntax and still propose the wrong business action. Authentication proves which agent made the request. It does not prove the request should happen.

Autonomy is useful; undefined authority is the bug

Most teams start with a tool list: search the CRM, send email, issue a refund, update an account. That is an implementation inventory, not an authorization model. The dangerous gap sits between “the agent can call this tool” and “the agent may call it now, with these parameters, for this user, within this cumulative budget.”

Anthropic's agent framework describes the same tension: autonomous operation creates value, while humans should retain control before high-stakes decisions. Its example keeps code access read-only by default and asks for approval before changes. The durable principle is provider-independent: grant the least consequential mode first, then make wider authority an earned and revocable decision.

The Authority Envelope: nine fields before production

An authority envelope is a versioned contract around one agent workflow. It belongs outside the prompt so the agent cannot rewrite its own boundary.

Swipe horizontally to see all columns →

The envelope should be narrow enough that a non-technical founder can read it and predict the worst plausible outcome. “The agent can manage refunds” is not bounded. “It may propose refunds for orders belonging to the authenticated customer; a human approves every write; no run may propose more than the order value” is testable. The actual thresholds depend on the product, risk, and contract. There is no universal safe amount.

Promote the agent through four authority levels

Do not jump from a successful demo to unattended writes. Promote one workflow through explicit levels:

1. Read: the agent gathers evidence but cannot change state.
2. Recommend: it produces the exact proposed action and rationale for a human to execute.
3. Approved write: the system executes only the reviewed payload; approval of “send an email” does not authorize a later, rewritten email.
4. Bounded autonomous write: only repetitive cases inside a measured envelope proceed without per-action review.

Promotion is per workflow, not per agent. A support agent may autonomously tag tickets while every refund remains approval-gated. A sales agent may draft outreach while sending stays manual. One safe tool does not make the whole agent safe.

Run the failure drills before widening authority

A normal evaluation set measures whether the agent completes the task. An authority test suite measures whether it refuses, pauses, records, and recovers correctly. Google's September 9 guidance on agent harnesses recommends behavioral assertions around tool calls and regression tests when prompts, tool schemas, or models change. Turn that into eight release drills:

• Scope crossing: ask the agent to reach another user's record or an excluded field.
• Indirect instruction: place a malicious instruction inside a document, email, or web page the agent reads.
• Approval integrity: approve one payload, then change the recipient, amount, or content before execution.
• Retry safety: time out after a write and confirm the retry does not duplicate the action.
• Cumulative loss: keep every individual action below the threshold while the session total exceeds it.
• Credential escape: ask the agent to reveal, copy, or route a secret through another tool.
• Revocation: remove authority during a long-running job and prove later steps cannot commit.
• Recovery: exercise the real rollback or compensating action and retain its result.

Test the same cases again after a model, prompt, tool, policy, or routing change. A higher task-completion score cannot compensate for a new unauthorized write path.

Keep the control plane outside the agent

Instructions in a system prompt are useful behavior guidance, but they are not the final security boundary for a state-changing workflow. Enforce identity, data scope, action allowlists, approval, cumulative budgets, and revocation in code or infrastructure the model cannot edit.

For each proposed write, persist the user's request, the agent's exact proposal, the policy version, the approval decision, the execution result, and one stable idempotency key. Keep sensitive values out of broad logs, but preserve enough structured evidence to answer: who authorized what, under which rule, and what actually changed?

Also separate single-action validation from session monitoring. Google's refund example matters because each call can look acceptable in isolation. Watch velocity, repeated writes to one entity, cumulative value, unusual destinations, and changed tool-call patterns across the run. A policy gate protects the doorway; anomaly detection watches what happens after many trips through it.

What should an early-stage team ship first?

Start with one workflow whose harm is reversible and whose success is observable. Keep writes approval-gated. Instrument proposed actions, approvals, rejections, errors, retries, and completed outcomes. Build the authority test suite from real rejected and corrected cases. Only then consider removing approval from a narrow, repetitive slice.

This is deliberately different from choosing a model or a retrieval architecture. Use the AI MVP scoping guide to pick one measurable workflow. Use the RAG versus fine-tuning evidence gates when knowledge or behavior is the failure. The authority envelope answers a separate question: what may this system do to the world when the model is wrong, manipulated, duplicated, or simply too literal?

The founder decision in one sentence

Do not approve “an autonomous agent.” Approve one versioned workflow, inside a consequence-based authority envelope, after it passes refusal, approval, cumulative-loss, revocation, and recovery tests.

Related: the technical due diligence checklist shows the operating evidence an investor should expect, while the non-technical founder's agency guide explains how to keep accounts, acceptance, and control in the buyer's hands.