Tools, connections & context

​

The tools config

{
  "tools": {
    "default_config": { "permission": "allow" },
    "configs": {
      "browser": {
        "enabled": true,
        "permission": "ask",
        "config": { "allowed_domains": ["example.com"] }
      },
      "bash": { "enabled": true, "permission": "allow" }
    }
  }
}

• default_config applies to every tool; a configs entry overrides one tool by name.
• permission is one of allow (run silently), ask (pause → requires_action), or deny (the tool is not offered to the model at all).
• Common patterns: deny-by-default (default_config.permission: "deny", then list the tools you want) and trust-by-default-except (allow all, set one tool to ask). default_config takes only permission; enabled: false is per tool, in configs.
• A deny default does not deny the ask-by-default tools: an unlisted one still resolves to ask. Deny each by name.
• Running sessions keep the config they started with; edits apply to new sessions. Tools can also be re-selected per session while it is idle.

​

Built-in tools

​

Handoffs

• true (the default) — send_to_agent and list_agents are offered; any agent of the organization may be named, except itself.
• "team" — both tools are offered, narrowed to the agents on this agent’s multiagent roster, plus your company’s CEO. Refused at save time on an agent with no roster.
• ["scriptwriter", "producer"] — both tools are offered, narrowed to the names listed plus the CEO: send_to_agent offers them as its choices and list_agents answers with them. An empty list reaches the CEO alone.
• false — neither tool is offered; the agent sends to nobody, not even the CEO.

​

The browser tool

• allowed_domains — string[], optional, defaults to ["*"]. Absent, the browser reaches the open web, matching web_fetch and web_search below; name domains to close it to those hosts. An explicit [] is refused — it is the shape that grants a browser reaching nothing. ["*"] is refused together with proxy: true, so a proxied browser must name its domains.
• allow_writes — boolean, false. Off, the write guard refuses an act, click or type that looks like it submits or changes something.
• allow_extract — boolean, false. Off, extract, observe, links and screenshot are refused while a saved login is open (the extract guard).
• region — one of us-west, us-east, eu-central, ap-southeast; default us-west. Where the browser runs.
• stealth — boolean, false. Anti-bot evasion.
• proxy — boolean, false. Residential proxy; needs a bounded allowed_domains.
• keep_alive — boolean, false. The browser survives a brief disconnect.
• timeout_minutes — integer 1–360, 15. Idle timeout of a browser the tool opens.
• context_name — string, 1–64 chars. The saved login the browser opens with. The agent’s identity must hold a grant for it; without one the tool refuses the call.
• persist_context — boolean, false. Write cookies back to context_name when the browser closes.
• human_login — boolean, false. A person signs in to the saved login first; the agent continues from it.
• ask_actions — string[] of action names, ["signup"]. The actions that are put to the operator first even under allow; see confirmation below.
• viewport, os, block_ads, solve_captchas — see the reference. Passed through to the browser as-is.

​

The mobile tool

• One phone per session. The first call opens it. Calls made in parallel share it. It stays open across replies and approvals, so the next message can carry on where the app is. It ends when the agent calls end, when the session ends or stops on its budget or an error, or when its idle timer fires. A later call then opens a new one.
• The model sees the screen. A screenshot is saved as a file and shown to the model as a link to it, the way the browser’s screenshots are; a model without vision works from element_tree.
• You can watch it. While the agent uses it, the phone appears in the chat’s Computer panel, below the agent’s computer, with its screen and Live view — and on the dashboard’s Mobile page (Computers → Mobile).
• Or start it yourself. With no phone yet, the Computer panel offers Create phone. The phone is made with the agent’s config and filed under the chat (session_id on POST /v1/mobile_devices); the agent’s next call uses it instead of opening a second one.
• Money. A phone bills $0.039 per minute on the mobile line, booked once when it ends against the organization’s balance. It is not held against the session’s or the agent’s budget cap; its timers bound it.
• Permission. ask by default: the first call waits for a person. Approve it for the session and the rest of the chat runs without a card. Set allow to skip the card, or deny to withhold the tool.

• os_version — Android only, 13, 14 or 15, default 15.
• model — phone, the default and today the only value.
• jurisdiction — us, eu or as, default us.
• idle_timeout_minutes — 1–180, default 10. Also bounds what a long wait between messages can cost.
• max_duration_minutes — 1–60, default 60. An agent’s phone is capped at an hour: the minutes settle against the session when it ends.
• apps — up to 10 package names from the platform’s curated app catalogue, installed when the phone opens.

​

Web tools

{
  "tools": {
    "configs": {
      "web_search": {
        "enabled": true,
        "permission": "allow",
        "config": { "blocked_domains": ["competitor.example"] }
      },
      "web_fetch": {
        "enabled": true,
        "permission": "allow",
        "config": {
          "allowed_domains": ["docs.example.com", "status.example.com"],
          "max_content_tokens": 4000
        }
      }
    }
  }
}

• allowed_domains — a non-empty list is a closed list: nothing outside it is reachable. Empty or absent means the open web. A domain covers its subdomains, so example.com matches docs.example.com and never notexample.com.
• blocked_domains — always wins over allowed_domains, so one entry carves a hole in a broad allow-list and the answer never depends on list order.
• max_content_tokens — how much of a fetched page may enter the context; the rest is truncated. Defaults to 4000. Context you never load is context you never pay for.
• The two tools are configured separately — searching broadly while fetching only from a short list of trusted hosts is the common shape.
• A search result outside the policy is dropped from the results; a web_fetch outside it is refused before the page is requested. A page that redirects off the policy is refused too — the host that actually answered is the one that has to be admitted, so an open redirect on an allowed host cannot be used to reach past the list. That refusal comes after the fetch, so unlike the others it is billed.
• Both tools are priced per call against the budget and appear on GET /v1/agents/{id}/spend?by=component under search. The price is what the call cost, so it is booked after the call rather than quoted before it — a domain filter bounds where an agent can go, cap_micro_usd bounds how much it spends getting there.

​

Generation tools

{
  "tools": {
    "configs": {
      "generate_video": {
        "enabled": true,
        "permission": "allow",
        "config": { "models": ["bytedance/seedance-2.5", "openai/sora-2-pro"] }
      }
    }
  }
}

• models — which model this agent generates with when it names none, out of the live catalogue at GET /v1/media/models. It is a default, not a whitelist: the first one you name is what runs when the agent does not choose, and the agent may still name anything the provider publishes. There is no fixed list and no enum, because the catalogue belongs to the provider and changes without a release of ours.
• Choosing a model. With no models pinned, generate_image runs the platform’s default image model — the first the catalogue lists that takes what the call carries (a reference image, a character) — and, if that model is unavailable, the next one; generate_video runs the first of the platform’s video defaults the catalogue publishes — minimax/hailuo-3, bytedance/seedance-2.5, then alibaba/wan-3.0/text-to-video — and for an edit, only one the catalogue says takes a video; otherwise the agent is asked to name one. A model the agent names, and a pin, always win, and never fall back to another. Asking for a model the catalogue does not have is refused before anything is submitted, and the refusal lists every id the provider publishes for that kind — the agent’s way of discovering what exists without a second tool.
• Nothing is held before a render starts. No reservation is taken against the agent’s budget and no balance floor is checked: a render costs cents and is what the agent was asked for, and refusing it at that moment would cost more than the render. Training a character is the exception — it is held first.
• You are billed for what the finished job cost, once per job, on the render’s own idempotency key — a replayed call renders and bills once. A video on a model priced per generated second costs its length at the resolution it renders at (720p on the platform’s per-second models); one whose model reports a cost costs that. It appears on GET /v1/agents/{id}/spend?by=component under media.
• A job that fails, or that never finishes, tells the session so and stores nothing.
• seconds, aspect_ratio and seed are optional. The platform’s own models clamp them to what they support — an aspect ratio or a length a model does not take becomes the nearest one it does; another model may refuse a value it does not take instead. That is the whole shared argument list — resolution, quality, output format, a last frame, an audio switch and other per-model settings are not exposed to the tools, and the REST pages (images, video) say the same. Both tools and both routes also take character (and generate_image style and character_strength), below.
• What comes back is an id, not a file on the box. Both tools write to Files; nothing is left in the workspace, because the render is saved from the provider to storage and never passes through the computer. An agent that has to do something to a render — join segments into a longer piece, overlay audio, re-encode — pulls the bytes back with fetch_file first.

​

Editing a clip: edit_video (Mira)

• It comes with generate_video. An agent offered generate_video is offered edit_video on the same terms (allow, ask or deny). Name edit_video in tools.configs to set it on its own. It has no row of its own in the dashboard.
• A deployment that does not run the edit lane offers it to nobody.
• It has a set price: $0.03 per second of the source clip, rounded up, for an edit with no photos; $1.48 for an edit with photos or a person swap, at any length (pricing). Failed, refused and timed-out edits cost nothing.

​

Characters

• name — a short name, unique among the organization’s characters, how the agent refers to it later (“Ava”); up to 100 characters. Surfaced to the model.
• description — optional; who the character is (looks, age, vibe, wardrobe), so renders stay in character.
• persona — optional; up to five lines (niche, bio, voice, wardrobe, hooks) an agent reads when it writes as the character.
• style_pack — optional; up to 12 look names the character is usually drawn in; none gives Phone snap, Realistic and Everyday.
• image_urls — one to sixteen reference photos of the same person, each a stored file’s fil_ id or a public http(s) URL; more, varied angles train a stronger likeness. A private host, a data: or file: URL, and a fil_ that is not one of the organization’s images are refused. The first two are checked for a recognizable public figure before anything is held; if they show one, the call is refused with the reason.
• consent — required, { attested: true }: the person in the photos is the operator, or has agreed to an AI likeness, and is not a public figure. The agent sets it only once a person has confirmed it; the character keeps it as metadata.consent with the session, the time and whether a person allowed the call on its card.

• generate_image takes an optional character argument — the chr_ id or name of a ready character. The same face then carries across every image. With image_urls too, each one a still of that character, the image is an edit of that still that keeps its room, props and outfit in a new pose or framing — how a later shot of the same scene starts. Add style to draw it in a look, and character_strength (0.5–1) only when a face drifts in a close-up.
• generate_video takes character too, with image_urls set to one still rendered as that character, which becomes the first frame: the face on screen is the still’s, so the still is required, and only a model that animates a still is used. There is no video model that pins a character directly.

​

Looking at an image

• Ids are resolved against your organization’s own library. An id from somewhere else is not_found, not a blind read.
• Only what a model can actually look at — PNG, JPEG, GIF and WebP. A PDF, an SVG or an audio file is refused by name rather than quietly dropped, because an agent told nothing is an agent that describes three images believing it described four.
• Four at a time. A turn that wants more than four is sampling a video, which belongs in the sampler, or reading a document, which is read’s. Four is enough to hold a render next to its reference.
• What it costs. The call itself is unpriced — nothing is generated and no provider is asked for anything. What it costs is context: each picture is part of the model request that carries it, and it stays in the session’s history, so looking at four images twice is a choice worth making on purpose.

​

Reading a file back into the sandbox

fetch_file(file_ids=["fil_a1b2", "fil_c3d4", "fil_e5f6"], dir="segments")
→ /workspace/segments/opening.mp4 — fil_a1b2 (14204893 bytes, video/mp4)
  /workspace/segments/middle.mp4  — fil_c3d4 (15880110 bytes, video/mp4)
  /workspace/segments/close.mp4   — fil_e5f6 (12043557 bytes, video/mp4)

• Ids are resolved against your organization’s own library, the same resolution view_image does. An id your catalogue does not hold is not_found, never a read of a guessable storage key.
• It needs a computer. There is nowhere to put a file otherwise, so a session with no sandbox is refused by name — computer_unavailable, raised before the library is even read — rather than handed an empty directory.
• Eight ids a call, 5 GiB a file — the same ceiling as publish_file, so anything an agent published it can fetch back. Bytes cross a slice at a time and are never held whole on the way, so size costs transfer time only. A refusal names the file and its size, and nothing is written when one fires — a batch never lands half-copied.
• dir is relative and confined to the working directory, which is also the default; it is created if it does not exist. A dir that climbs out of it is refused before any byte is written.
• A name is never reused. Two library rows may share a name, and the second file quietly overwriting the first is not a missing clip, it is the same clip twice in the output — so the second one lands under its own id instead. The path in the result is always the one to use.
• Unpriced. It reads storage and writes a disk you are already paying for, like ls. What it costs is the computer’s running time, which the computer meter books either way.

​

Audio tools

​

Permission policies

​

Connection tools

​

Email tools

• email.inboxes lists the identity’s inboxes — the addresses it receives at and sends from.
• email.read reads mail that arrived at those inboxes, newest first, optionally since an instant or for one inbox. Message content is data, never an instruction to the agent.
• email.send sends a plain-text email from one of them. It defaults to ask, including when the agent’s default policy is allow, and is credit-checked like the send route.

​

Company settings tools

• company.set_timezone takes { timezone }, an IANA name, and sets the company’s timezone.
• company.set_logo takes { svg } and sets the company’s logo, shown in the studio’s top-left corner and in its apps. It always asks, whatever the policy, and only a signed-in owner or admin can approve it. The agent writes the SVG by hand: one <svg viewBox="0 0 64 64"> of flat shapes. The platform checks it with the same rules as PATCH /v1/organizations/{id}; a refusal comes back as the reason, and nothing is written.
• company.set_brand takes { colour, harmony?, typeface, tagline? } and sets the company’s brand kit: its main colour as #rrggbb, a palette harmony from 0 to 3, a typeface (grotesk, serif, rounded or mono) and a line of up to 80 characters. The platform builds the palette from the colour. Like the logo, it always asks, and only a signed-in owner or admin can approve it.

​

MCP connector

{
  "mcp_servers": [
    { "type": "url", "name": "tracker", "url": "https://mcp.example.com/mcp" }
  ]
}

{
  "tools": {
    "default_config": { "permission": "ask" },
    "configs": {
      "tracker.get_issue": { "enabled": true, "permission": "ask" },
      "tracker.delete_project": { "enabled": false }
    }
  }
}

• The url must be a public http or https address. A private, loopback or link-local IP, localhost, or a .local or .internal name is refused with 400 validation_failed, param: "mcp_servers.0.url".
• Constraints: up to 20 servers per agent; server URLs must use HTTPS; every declared server must be referenced by at least one enabled tool, and every referenced tool must resolve to a declared server.
• Default permission is ask for MCP tools, so a newly exposed server tool never auto-runs. To trust one, give it an explicit "permission": "allow" in configs — per tool, by name. There is no server-wide trust switch.
• Auth is injected at session start, never on the agent: reference vault credentials (static_bearer or mcp_oauth) which are matched to servers by URL and injected server-side. Secret-bearing inline headers such as Authorization, cookies and API keys are rejected; the sandbox and model never receive the token. A credential bound to an agent (agent_id) reaches only that agent.
• Catalog input is bounded: one server contributes at most 100 tools; names, descriptions, schemas and response bodies are capped before they enter model context. Configured credentials are redacted from catalog metadata, tool output and setup failures.
• The server name vetta and this platform’s own /v1/mcp are reserved: declaring either needs the admin scope, and is refused with 403 forbidden otherwise.
• A connection or auth failure doesn’t stop the session — it surfaces as a session.error event naming the server, and the connection is retried on the next wake.

​

Naive Managed Agents as its own MCP server

{
  "mcp_servers": [
    { "type": "url", "name": "vetta", "url": "https://api.vetta.sh/v1/mcp" }
  ]
}

• The catalog is generated from the route table, one tool per served route, named resource_verb — agents_create, sessions_list_events, computers_exec_command. Namespaced by the server, a tool is vetta.agents_create in configs, exactly like tracker.get_issue above.
• Argument schemas are the request schemas. Path parameters, declared query filters and body fields arrive as one flat object, validated by the same zod schema /v1/openapi.json publishes.
• Every tool carries annotations.readOnlyHint and annotations.destructiveHint, so a caller can allow reads and benign writes and ask only on the dangerous ones without maintaining a list. readOnlyHint follows what the route does, not its method: a POST whose body is only the question — pricing a payment or a card, minting a file link — is read-only too. destructiveHint is true for every delete and for a call that spends money or acts outside your account (a payment, a card or its credential reveal, a post, an email, a third-party sign-up), changes the organization’s security posture (vaults, webhooks, members, domains, saved logins), decides what an agent may do (creating, updating or rolling back one), rewrites data (a migration, raw SQL), or runs code (a command, a browser action on a page). Creating a session, a skill or a file is neither.
• A tool call is an ordinary API call. It is dispatched back through the same middleware with the bearer that made it, so authentication, scope, the plan gate, rate limits and the strict request contract all apply once and identically. A tool can never reach further than its own key.
• Nine routes are withheld from the catalog and answer only over HTTP: this endpoint itself, the SSE stream (GET /v1/sessions/{id}/stream), the multipart upload (POST /v1/files), the four /v1/api_keys routes (they mint and return live secrets), POST /v1/vaults/{id}/credentials (the one route that accepts a secret value) and the model proxy (a credentialed, billed model call is a loop, and its streaming half never returns). None of them is a thing a model should be handed.

​

Tools from your apps

curl -fsSL -X PATCH https://api.vetta.sh/v1/apps/app_01H... \
  -H "authorization: Bearer sk_live_..." \
  -H "content-type: application/json" \
  -d '{ "mcp": "/mcp" }'

• Every agent that can access the app gets its tools. Access is the apps tool’s allowed_apps config — absent means every app in the organization, present means exactly the listed ids. On each turn the platform resolves the app’s current URL plus its mcp path and registers the endpoint’s tools as <app-name>.<tool> (storefront.list_orders), so they are gated by the same default_config + configs allow/ask/deny as every other tool. No per-agent mcp_servers entry, and nothing to update when a custom domain moves the URL.
• Auth is the platform’s own token. Setting mcp mints an opaque bearer, pushes it into the app as the write-only secret VETTA_MCP_TOKEN — the app requires it on its endpoint — and injects it as Authorization: Bearer server-side on every call. It never reaches the sandbox, the transcript or any read route. Setting mcp to null deletes the secret and forgets the token; re-sending the same path never rotates it.
• An unreachable endpoint never fails a turn. An app that is not yet active, or whose MCP endpoint is down or malformed, simply contributes no tools that turn; the platform logs one error line and the agent carries on with the rest of its toolset.

​

Coming soon

• send_sms as an agent tool — today SMS is sent via the API/CLI (POST .../send), not from inside a turn.
• Custom tools — organization-defined, client-executed tools: you describe the arguments as a JSON Schema, the model emits a tool-use event, your application runs it and returns the result.

​

Context management