How to add human approval to an AI agent

Gate an AI agent's risky tool calls with an ask policy: the session pauses, a person approves it over a webhook, the dashboard or the CLI, and a USD budget caps spend regardless.

Hold the call, do not ask the model

Human approval for an AI agent means a sensitive action stops and waits for a person. Issue the refund, delete the project, send the mail: none of it happens until someone says so. The unsafe way to get this is to ask the model in its system prompt to "check with a human first". The safe way is for the platform to hold the call itself, so the policy holds regardless of what the model decides to attempt.

On Vetta that hold lives at the tool-call boundary. This guide covers how a tool call resolves to allow, ask or deny, what an ask does to a running session, how a person answers it, and why a USD budget sits underneath all of it. It replaces our older posts on spend caps, budget caps, agent permissions and governance, which now redirect here.

Where approval lives: the tool-call boundary

Every capability an agent has flows through a tool call, so that is where Vetta checks policy, before the tool runs rather than in an audit afterwards. Each tool resolves to one of three permissions:

The resolved permission comes from two layers, most specific wins: an organization default that every agent inherits, and a per-agent, per-tool override. Tools are configured as a single toolset: a default_config that applies to every tool, and per-tool configs that enable a tool and override its permission.

allow is convenient, but it is not automatically the right default. For anything irreversible or externally visible (payouts, deletes, outbound mail) set ask. For anything the agent should never do, set deny rather than leaning on the permissive baseline. The Approvals primitive is this mechanism; the policies docs are the reference.

Step 1: mark the risky tools ask

An agent is a declarative configuration, so the cleanest place to set permissions is the agent's checked-in .agent.yaml:

name: Refunder
model:
  id: zai-org/GLM-5.2-FP8
harness: pi
system: |
  You process refunds. Always confirm the order first, then apply the
  refund-policy skill. Escalate anything ambiguous rather than guessing.
skills:
  - refund-policy
tools:
  default_config:
    permission: allow
  configs:
    bash:    { enabled: true, permission: ask }
    browser: { enabled: true, permission: ask }
budget:
  cap_usd: 50
  max_task_usd: 5
  period: month

vetta agent apply -f refunder.agent.yaml

To flip one tool on an agent that already exists, without touching the rest of its config:

vetta agent tools Refunder --tool bash --permission ask

Connection and MCP tools take the same treatment, keyed <connector>.<tool>, for example tracker.get_issue. Some identity actions are approval-gated out of the box: connecting a new third-party app, provisioning a phone number and purchasing a domain all hold for a person before anything external happens.

An agent update mints a new immutable version rather than editing the current one, and a session's tool set is fixed at creation from the agent's configuration. To change permissions on a running session, update the agent and start a new one.

Step 2: what the agent hits

When the agent calls a tool set to ask, the runtime does not run it. It streams a tool.confirm event and parks the session: status goes idle with stop_reason: "awaiting_approval", and the held call, with its tool_call_id, name and args, is exposed on pending_actions[].

{
  "id": "ses_4a...",
  "status": "idle",
  "stop_reason": "awaiting_approval",
  "pending_actions": [
    {
      "kind": "tool",
      "tool_call_id": "call_8Q...",
      "name": "refund",
      "args": { "order_id": "4821", "amount_micro_usd": 42000000 }
    }
  ]
}

A held tool consumes no budget while it waits. The session loop is durable (wake, take a turn, commit, sleep), so an agent can sit on a confirmation for hours at storage cost only. That is what makes it acceptable to gate aggressively.

Step 3: get the pending call in front of a person

An idle session that is waiting on you looks exactly like one that finished its work; status cannot tell them apart. stop_reason can, so every surface filters on it.

Webhook. Subscribe an endpoint to session.idle. Each delivery is a signed JSON envelope whose data carries the session_id and stop_reason, so your handler branches on awaiting_approval and fetches the session for the pending call. Verify Vetta-Signature against the raw body first, and reject deliveries whose Vetta-Timestamp is more than 300 seconds old.

vetta webhook add \
  --url https://example.com/hooks/vetta \
  --events session.idle,budget.exceeded

// `verify` is the HMAC-SHA256 check from the webhooks docs; the SDK does not ship one.
app.post("/hooks/vetta", express.raw({ type: "application/json" }), async (req, res) => {
  if (!verify(req.body, req.headers, process.env.VETTA_WEBHOOK_SECRET!)) return res.sendStatus(400);
  const event = JSON.parse(req.body.toString("utf8"));
  if (event.type === "session.idle" && event.data.stop_reason === "awaiting_approval") {
    const session = await client.sessions.get(event.data.session_id);
    await notifyApprovers(session.id, session.pending_actions);
  }
  res.sendStatus(200);
});

CLI. vetta session inbox lists every session waiting on a person and what each one wants, and vetta session list takes a --stop-reason filter for scripts:

vetta session inbox --human
vetta session list --stop-reason awaiting_approval --json | jq -r '.data[].id'

SDK. The same filter is a query on sessions.list:

const waiting = await client.sessions.list({ stop_reason: "awaiting_approval" });

Dashboard. The session page shows one card per held call, with the tool name, its arguments and approve or deny buttons. A product that embeds agents for its own customers forwards the webhook into its own approval UI instead.

Step 4: approve or deny

A confirmation is a verb and no payload: allow or deny, optionally with a reason on a deny. From the CLI:

vetta session confirm --session $SID --tool-call call_8Q... --allow
vetta session confirm --session $SID --tool-call call_8Q... --deny --reason "not this account"

From the SDK, walk pending_actions and answer each held tool:

const s = await client.sessions.get(sessionId);
for (const a of s.pending_actions) {
  if (a.kind !== "tool") continue; // a question is answered, not approved
  await client.sessions.confirmTool(sessionId, {
    tool_call_id: a.tool_call_id,
    decision: a.name === "refund" ? "allow" : "deny",
    reason: "out of policy",
  });
}

Branch on kind, not on stop_reason alone. pending_actions[] also carries kind: "question" rows when the agent parked itself on a question for you; those take an answer through sessions.answer, and sending them a confirmation is a validation error.

Once the last pending action is answered the session resumes automatically. On allow the tool runs and the agent continues. On deny the tool never runs and the deny message is fed back to the agent in-band, so it can plan around the refusal instead of crashing. Every resolved confirmation records the approving principal, the user or API key id that answered, on the event and in the audit log alongside the decision and any reason. That is what lets you prove afterwards that a payout was signed off by a person.

Do not let an approval hang forever

An ask tool does not hold the turn indefinitely. Two fields on the policy bound the wait: ask_timeout_seconds sets how long a call may sit unanswered, and on_timeout decides what happens when it elapses.

Omit the timeout and the session waits indefinitely, at storage cost. For an unattended overnight run, deny is almost always right.

Budgets cap spend regardless

Approval decides which actions may happen. A budget decides how much they may cost, and it applies whether or not a person is watching. An agent cannot be created without one: a period cap (cap_micro_usd), a per-task ceiling (max_task_micro_usd) and a reset period of day, week or month. Money is integer micro-USD on the wire; the CLI's --budget-usd and --max-task-usd (and cap_usd / max_task_usd in .agent.yaml) take decimal dollars and convert client-side.

vetta agent create --name nightly-triage --model zai-org/GLM-5.2-FP8 \
  --budget-usd 50 --max-task-usd 5 --budget-period month

Before each model call or priced tool call, the runtime computes a quote, an upper bound at the session's completion window, and checks it in order against the organization balance, the agent's period cap, the task ceiling and any session budget. If the quote would breach any of them the call is refused, budget.exceeded is emitted, and the agent is told in-band so it can wrap up cleanly. A session can carry its own cap too, and work that pauses at it resumes when you raise it:

vetta session create --agent Refunder --budget-usd 2.00
vetta session budget --session $SID --usd 5.00   # raise; must exceed what is already consumed

The two mechanisms compose. An identity policy can attach a spend threshold to an approval rule, so an action whose quoted cost exceeds the amount forces a human decision even when the primitive is otherwise allowed. The budget caps the total; the policy governs which actions may spend at all. A budget_paused session shows up in vetta session inbox next to the approvals, because both are waiting on a person. See the Budgets primitive and the budgets docs.

How you know it worked

• The agent calls an ask tool and you see a tool.confirm event, then session.idle with stop_reason: "awaiting_approval". Nothing external happened.
• Allow: the tool runs, tool.completed follows, and the session continues on its own.
• Deny: no side effect, and the agent receives the deny message in-band.
• The confirmation event names the principal who answered.
• A call that would breach the cap is refused with budget.exceeded before it runs, approved or not.

Where to go next

Wire one tool first: set it to ask, drive a session into it, answer from the CLI, and watch the stream. Then add the webhook so approvals reach the people who own the decision.

Approval is one half of trust; the other half is taking access away. How to revoke AI agent access instantly covers that side. If you are embedding agents in a product your own customers use, Building AI agents into your SaaS is the companion read. The reference material lives at Policies, Session operations, Events and streaming and Webhooks.