> ## Documentation Index
> Fetch the complete documentation index at: https://vetta.sh/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Model router & inference
> How Vetta routes each call to inference, what the completion window changes about it, and how model tokens are metered.
Every model call an agent makes goes through Vetta's **model router**. The router picks the inference backend for each call based on two inputs — the **model** and the [**completion window**](/docs/concepts/completion-window) — meters the tokens against your [budget](/docs/concepts/budgets) before the call runs, and returns the result. The [harness](/docs/how-vetta-is-built#the-three-layers) decides *when* to call the model; the router — part of the [runtime](/docs/concepts/runtime) — decides *where the call goes* and *what it costs*.
You never address a backend directly. You name a model and a window; the router does the rest.
## Two inference lanes
Vetta federates two inference backends behind one interface:
* **Aggregated inference network** — a broad catalogue of models available at interactive latency. This is the default lane and serves every `immediate` request.
* **Completion-window pool** — specialized open-weights hosting that offers genuine reduced tariffs for the `priority` and `loose` windows at reduced latency cost. This lane serves `priority` and `loose` requests, and it only hosts a **specific set of window-supported models**.
```
model call (model + window)
│
├─ window = immediate ─────────────▶ Aggregated inference network
│ any model in the catalogue · immediate tariff
│
└─ window = priority | loose ──────▶ Completion-window pool
│ priority / loose tariff
│
└─ model NOT window-supported ─▶ 400, refused before any spend
```
## The routing rule
The window determines the lane, and the lane constrains the model:
| Window | Lane | Model requirement |
| ----------- | ---------------------- | --------------------------------------------------- |
| `immediate` | Aggregated network | Any model in the catalogue, including `vetta/auto`. |
| `priority` | Completion-window pool | Must be a **window-supported** model. |
| `loose` | Completion-window pool | Must be a **window-supported** model. |
A **non-default window** (`priority` or `loose`) with a model that the completion-window pool does not host is refused with a typed error — `window_unavailable`, **HTTP 400** — before any inference runs and before any spend. `immediate` always works, on any catalogued model. See [Errors](/docs/api/errors).
This is a deliberate fail-closed: rather than silently downgrading a `loose` request to the interactive tariff (and quietly overcharging you), the router rejects the combination so you fix it explicitly — either pick a window-supported model, or drop to `immediate`.
## The catalogue is live
There is **no curated list**. The catalogue is read from the inference network at request time and
cached briefly, then filtered down to the models Vetta can actually drive an agent with:
* text out — a model that returns images or audio is not an agent's model;
* tool calling — the harness cannot run a loop with a model that cannot call a tool;
* a published per-token price for both prompt and completion, because a model that cannot be quoted
cannot be metered, and metering before the call is what makes spend fail closed.
That is **hundreds of models**, and it moves on its own: a model the network publishes today is
runnable today, with no release of ours. Two consequences follow for anything that reads it — the
listing is **paged**, and it is **searchable**. Nothing should assume one call returns the whole
catalogue, and nothing should hard-code a model id it has not checked.
## Discovering models
`GET /v1/models` is the catalogue; `GET /v1/models/{id}` is one entry, for the id you already hold.
```bash CLI theme={"system"}
vetta models list --search glm --limit 5
vetta models list --window priority # only what that window can serve
vetta models get zai-org/GLM-5.2-FP8
```
```bash API theme={"system"}
curl -fsSL "https://api.vetta.sh/v1/models?search=glm&limit=5" \
-H "authorization: Bearer sk_live_..."
# Model ids contain a slash — percent-encode the path segment.
curl -fsSL "https://api.vetta.sh/v1/models/zai-org%2FGLM-5.2-FP8" \
-H "authorization: Bearer sk_live_..."
```
```typescript TypeScript theme={"system"}
const page = await vetta.models.list({ search: "glm", limit: 5 });
const more = await vetta.models.list({ after: page.next_cursor });
const m = await vetta.models.retrieve("zai-org/GLM-5.2-FP8");
console.log(m.supported_windows); // ["immediate","priority","loose"]
```
| Query | What it does |
| -------- | ------------------------------------------------------------------------------------------------ |
| `window` | Narrow to the models that window can actually be served in — `immediate`, `priority` or `loose`. |
| `search` | Free text, matched against the model's id and name. |
| `limit` | Page size, 1–100. Defaults to 20. |
| `after` | The previous page's `next_cursor`. |
The listing is [cursor-paged](/docs/api/pagination) like every other list on the API: `has_more` and
`next_cursor` are real, not constants. An id the deploy does not serve is a `not_found` (404) from
the by-id route — which makes it the cheapest way to validate a model id before you run on it.
```json theme={"system"}
{
"object": "model",
"id": "zai-org/GLM-5.2-FP8",
"context_window": 1048576,
"max_output_tokens": 32768,
"supported_windows": ["immediate", "priority", "loose"],
"efforts": ["low", "medium", "high"]
}
```
The model object publishes **no prices** — the same line [`GET /v1/media/models`](/docs/api/media) draws.
What you spend is bounded by the agent's [budget](/docs/concepts/budgets) before the call, and read back
as actuals from [`vetta agent spend`](/docs/cli/agents#spend). A published rate card would be a number to
reconcile against; the ledger is the number that is true.
`?window=` narrows to what a window can serve — the same derivation the router refuses on, so a
model listed for a window is never rejected for it.
## `max_output_tokens`
Every entry publishes `max_output_tokens`: the longest reply that model may produce, taken from what
the model itself advertises and clamped to a platform ceiling. It is **per model**, not one number
for the fleet — a model that can write 131 072 tokens and one that can write 8 192 are not bounded
the same way.
It is also what the **pre-flight quote is bounded by**. Before a call runs, the router reserves the
worst case: every input token at the input rate, plus `max_output_tokens` at the output rate. The
call then settles at what it actually used and the remainder is released. So `max_output_tokens`
sets how much of a [budget](/docs/concepts/budgets) one in-flight call reserves, not what it costs — a
long-output model holds more credit while it is running, and returns the difference when it is done.
## Vetta Auto
`vetta/auto` is a model id like any other, and it picks the model per request: you name the task,
the router picks the model that fits it, call by call. It is useful when a workload is uneven — a
mix of trivial and hard turns — and you would rather not pin one model expensive enough for the
worst of them.
```typescript TypeScript theme={"system"}
const agent = await vetta.agents.create({
name: "Triage",
model: "vetta/auto",
budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" },
});
```
Two things are specific to it:
* **`immediate` only.** It routes across the aggregated network, so it never runs in the
completion-window pool. `priority` or `loose` with `vetta/auto` is refused with
`window_unavailable`, exactly like any other unsupported pair.
* **It is priced as a ceiling, then settled at the model that answered.** Because the model is not
known until the call is routed, there is no rate card to quote from. So the request carries a
**hard price cap** — \$5 per million input tokens and \$25 per million output tokens — which the
network enforces: a call it cannot serve inside the cap is refused rather than routed to something
dearer. The pre-flight hold is taken at exactly those cap rates, so it is a genuine upper bound.
The debit is then settled at the **rate of the model that actually answered**, which is normally
well below the cap. You are never billed above the ceiling you were quoted.
`vetta/auto` selects a different model for different requests by design. Pin a specific id instead
when a run has to be reproducible, or when a prompt is tuned to one model's behaviour.
## What a model call costs
Model spend is metered per token across a **five-tier ledger** — each class of token is priced separately because the backends bill them separately. The per-token rate is set by the request's [completion window](/docs/concepts/completion-window) tariff, and that rate is what you are charged.
| Token tier | Field | What it is |
| --------------- | ------------- | ------------------------------------------------------------------------- |
| **Input** | `input` | Fresh prompt tokens sent to the model. |
| **Cache write** | `cache_write` | Prompt tokens written into the provider's prompt cache. |
| **Cache read** | `cache_read` | Prompt tokens served from cache — billed at a fraction of the input rate. |
| **Output** | `output` | Tokens the model generates. |
| **Reasoning** | `reasoning` | Internal thinking tokens, when the model and `effort` produce them. |
These five fields are the canonical token ledger: every inference debit records all five, and they are the components a [session's](/docs/concepts/sessions#cost-and-usage) `token_usage` and the billing line items are built from.
Cache-read tokens are the reason list-price rate cards mislead. A card that bills every input token at the full input rate over-states real cost by **1.017×–4.176×** depending on harness and window (see [Benchmarks](https://usenaive.ai/benchmarks)). Vetta meters each tier at its real rate, so your bill tracks what the backend actually charged — not a rate-card fiction.
Because every call is priced against these five tiers *before* it runs, a call that would breach your [budget](/docs/concepts/budgets) is refused rather than discovered on an invoice. Model spend shows up as the `model` line item in the agent's spend breakdown:
```bash CLI theme={"system"}
vetta agent spend Refunder --by component
```
```json theme={"system"}
{ "by_component": { "model": 11902000, "computer": 481000, "search": 6474, "media": 390000 } }
```
Amounts are integer micro-USD (`11902000` is \$11.902). The `model` component is the sum of the five token tiers above; `computer`, `search` and `media` are the other components a debit can carry, and a component with no spend is absent rather than zero.
## Effort
Some models accept an `effort` level that trades latency and reasoning-token spend for quality. Effort is independent of the completion window: the window sets the *tariff and latency lane*, effort sets *how hard the model thinks within it*.
Which levels a model accepts is published per model in `efforts`, read from what the model itself advertises and narrowed to the three wire values. An empty `efforts` means the model takes no effort setting at all — most do not — so check the entry before you pin one.
```typescript TypeScript theme={"system"}
const agent = await vetta.agents.create({
name: "Deep",
model: { id: "zai-org/GLM-5.2-FP8", effort: "high" },
window: "priority",
budget: { capMicroUsd: "25000000", maxTaskMicroUsd: "3000000", period: "month" }, // $25 / $3
});
```
## Configuration reference
A model ID (e.g. `zai-org/GLM-5.2-FP8`, or `vetta/auto` to pick per request) or an object `{ id, effort }`. Set on the agent and overridable per [session](/docs/concepts/sessions#override-agent-configuration-for-a-session). Any id [`GET /v1/models`](#discovering-models) publishes is legal; anything else is `validation_failed`.
The model identifier.
`low`, `medium`, or `high`, for models that support it.
The [completion window](/docs/concepts/completion-window). `immediate` routes to the aggregated network; `priority` and `loose` route to the completion-window pool and require a window-supported model.
### Read-only model fields
Maximum context length in tokens.
The longest reply this model may produce, and what the [pre-flight quote](#max_output_tokens) is bounded by.
Which windows the model can run in. `immediate` is always present; `priority`/`loose` appear only for pool-hosted models.
Effort levels the model accepts — `low`, `medium`, `high`, or empty when it accepts none.
The three-price model, and why you choose it per request.