Model router & inference

Every model call an agent makes goes through Vetta’s model router. The router picks the inference backend for each call based on two inputs — the model and the completion window — meters the tokens against your budget before the call runs, and returns the result. The harness decides when to call the model; the router — part of the runtime — decides where the call goes and what it costs. You never address a backend directly. You name a model and a window; the router does the rest.

Two inference lanes

Vetta federates two inference backends behind one interface:

Aggregated inference network — a broad catalogue of models available at interactive latency. This is the default lane and serves every immediate request.
Completion-window pool — specialized open-weights hosting that offers genuine reduced tariffs for the priority and loose windows at reduced latency cost. This lane serves priority and loose requests, and it only hosts a specific set of window-supported models.

 model call  (model + window)
        │
        ├─ window = immediate ─────────────▶ Aggregated inference network
        │                                     any model in the catalogue · immediate tariff
        │
        └─ window = priority | loose ──────▶ Completion-window pool
                       │                      priority / loose tariff
                       │
                       └─ model NOT window-supported ─▶ 400, refused before any spend

The routing rule

The window determines the lane, and the lane constrains the model:

Window	Lane	Model requirement
`immediate`	Aggregated network	Any model in the catalogue, including `vetta/auto`.
`priority`	Completion-window pool	Must be a window-supported model.
`loose`	Completion-window pool	Must be a window-supported model.

A non-default window (priority or loose) with a model that the completion-window pool does not host is refused with a typed error — window_unavailable, HTTP 400 — before any inference runs and before any spend. immediate always works, on any catalogued model. See Errors.

This is a deliberate fail-closed: rather than silently downgrading a loose request to the interactive tariff (and quietly overcharging you), the router rejects the combination so you fix it explicitly — either pick a window-supported model, or drop to immediate.

The catalogue is live

There is no curated list. The catalogue is read from the inference network at request time and cached briefly, then filtered down to the models Vetta can actually drive an agent with:

text out — a model that returns images or audio is not an agent’s model;
tool calling — the harness cannot run a loop with a model that cannot call a tool;
a published per-token price for both prompt and completion, because a model that cannot be quoted cannot be metered, and metering before the call is what makes spend fail closed.

That is hundreds of models, and it moves on its own: a model the network publishes today is runnable today, with no release of ours. Two consequences follow for anything that reads it — the listing is paged, and it is searchable. Nothing should assume one call returns the whole catalogue, and nothing should hard-code a model id it has not checked.

Discovering models

GET /v1/models is the catalogue; GET /v1/models/{id} is one entry, for the id you already hold.

vetta models list --search glm --limit 5
vetta models list --window priority          # only what that window can serve
vetta models get zai-org/GLM-5.2-FP8

curl -fsSL "https://api.vetta.sh/v1/models?search=glm&limit=5" \
  -H "authorization: Bearer sk_live_..."

# Model ids contain a slash — percent-encode the path segment.
curl -fsSL "https://api.vetta.sh/v1/models/zai-org%2FGLM-5.2-FP8" \
  -H "authorization: Bearer sk_live_..."

const page = await vetta.models.list({ search: "glm", limit: 5 });
const more = await vetta.models.list({ after: page.next_cursor });
const m = await vetta.models.retrieve("zai-org/GLM-5.2-FP8");
console.log(m.supported_windows); // ["immediate","priority","loose"]

Query	What it does
`window`	Narrow to the models that window can actually be served in — `immediate`, `priority` or `loose`.
`search`	Free text, matched against the model’s id and name.
`limit`	Page size, 1–100. Defaults to 20.
`after`	The previous page’s `next_cursor`.

The listing is cursor-paged like every other list on the API: has_more and next_cursor are real, not constants. An id the deploy does not serve is a not_found (404) from the by-id route — which makes it the cheapest way to validate a model id before you run on it.

{
  "object": "model",
  "id": "zai-org/GLM-5.2-FP8",
  "context_window": 1048576,
  "max_output_tokens": 32768,
  "supported_windows": ["immediate", "priority", "loose"],
  "efforts": ["low", "medium", "high"]
}

The model object publishes no prices — the same line GET /v1/media/models draws. What you spend is bounded by the agent’s budget before the call, and read back as actuals from vetta agent spend. A published rate card would be a number to reconcile against; the ledger is the number that is true.

?window= narrows to what a window can serve — the same derivation the router refuses on, so a model listed for a window is never rejected for it.

`max_output_tokens`

Every entry publishes max_output_tokens: the longest reply that model may produce, taken from what the model itself advertises and clamped to a platform ceiling. It is per model, not one number for the fleet — a model that can write 131 072 tokens and one that can write 8 192 are not bounded the same way. It is also what the pre-flight quote is bounded by. Before a call runs, the router reserves the worst case: every input token at the input rate, plus max_output_tokens at the output rate. The call then settles at what it actually used and the remainder is released. So max_output_tokens sets how much of a budget one in-flight call reserves, not what it costs — a long-output model holds more credit while it is running, and returns the difference when it is done.

Vetta Auto

vetta/auto is a model id like any other, and it picks the model per request: you name the task, the router picks the model that fits it, call by call. It is useful when a workload is uneven — a mix of trivial and hard turns — and you would rather not pin one model expensive enough for the worst of them.

TypeScript

const agent = await vetta.agents.create({
  name: "Triage",
  model: "vetta/auto",
  budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" },
});

Two things are specific to it:

immediate only. It routes across the aggregated network, so it never runs in the completion-window pool. priority or loose with vetta/auto is refused with window_unavailable, exactly like any other unsupported pair.
It is priced as a ceiling, then settled at the model that answered. Because the model is not known until the call is routed, there is no rate card to quote from. So the request carries a hard price cap — $5 per million input tokens and $25 per million output tokens — which the network enforces: a call it cannot serve inside the cap is refused rather than routed to something dearer. The pre-flight hold is taken at exactly those cap rates, so it is a genuine upper bound. The debit is then settled at the rate of the model that actually answered, which is normally well below the cap. You are never billed above the ceiling you were quoted.

vetta/auto selects a different model for different requests by design. Pin a specific id instead when a run has to be reproducible, or when a prompt is tuned to one model’s behaviour.

What a model call costs

Model spend is metered per token across a five-tier ledger — each class of token is priced separately because the backends bill them separately. The per-token rate is set by the request’s completion window tariff, and that rate is what you are charged.

Token tier	Field	What it is
Input	`input`	Fresh prompt tokens sent to the model.
Cache write	`cache_write`	Prompt tokens written into the provider’s prompt cache.
Cache read	`cache_read`	Prompt tokens served from cache — billed at a fraction of the input rate.
Output	`output`	Tokens the model generates.
Reasoning	`reasoning`	Internal thinking tokens, when the model and `effort` produce them.

These five fields are the canonical token ledger: every inference debit records all five, and they are the components a session’s token_usage and the billing line items are built from.

Cache-read tokens are the reason list-price rate cards mislead. A card that bills every input token at the full input rate over-states real cost by 1.017×–4.176× depending on harness and window (see Benchmarks). Vetta meters each tier at its real rate, so your bill tracks what the backend actually charged — not a rate-card fiction.

Because every call is priced against these five tiers before it runs, a call that would breach your budget is refused rather than discovered on an invoice. Model spend shows up as the model line item in the agent’s spend breakdown:

CLI

vetta agent spend Refunder --by component

{ "by_component": { "model": 11902000, "computer": 481000, "search": 6474, "media": 390000 } }

Amounts are integer micro-USD (11902000 is $11.902). The model component is the sum of the five token tiers above; computer, search and media are the other components a debit can carry, and a component with no spend is absent rather than zero.

Effort

Some models accept an effort level that trades latency and reasoning-token spend for quality. Effort is independent of the completion window: the window sets the tariff and latency lane, effort sets how hard the model thinks within it. Which levels a model accepts is published per model in efforts, read from what the model itself advertises and narrowed to the three wire values. An empty efforts means the model takes no effort setting at all — most do not — so check the entry before you pin one.

TypeScript

const agent = await vetta.agents.create({
  name: "Deep",
  model: { id: "zai-org/GLM-5.2-FP8", effort: "high" },
  window: "priority",
  budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" }, // $25 / $3
});

Configuration reference

string | object

required

A model ID (e.g. zai-org/GLM-5.2-FP8, or vetta/auto to pick per request) or an object { id, effort }. Set on the agent and overridable per session. Any id GET /v1/models publishes is legal; anything else is validation_failed.

Show model object

string

required

The model identifier.

string

low, medium, or high, for models that support it.

string

default:"immediate"

The completion window. immediate routes to the aggregated network; priority and loose route to the completion-window pool and require a window-supported model.

Read-only model fields

integer

Maximum context length in tokens.

integer

The longest reply this model may produce, and what the pre-flight quote is bounded by.

string[]

Which windows the model can run in. immediate is always present; priority/loose appear only for pool-hosted models.

string[]

Effort levels the model accepts — low, medium, high, or empty when it accepts none.

Next: completion window

The three-price model, and why you choose it per request.

Agent

Running an agent

Agent capabilities

Computer

Identity

Team

Model router & inference

Two inference lanes

The routing rule

The catalogue is live

Discovering models

`max_output_tokens`

Vetta Auto

What a model call costs

Effort

Configuration reference

Read-only model fields

Next: completion window

​Two inference lanes

​The routing rule

​The catalogue is live

​Discovering models

​max_output_tokens

​Vetta Auto

​What a model call costs

​Effort

​Configuration reference

​Read-only model fields

Next: completion window

Two inference lanes

The routing rule

The catalogue is live

Discovering models

`max_output_tokens`

Vetta Auto

What a model call costs

Effort

Configuration reference

Read-only model fields