Benchmarks.Measured, not modelled.
Media
Our own benches. A model runs a short-form media business end to end and is scored on what it has at the end of the season.
Clipping Bench v1.0
~30 days seasonModels run a short-form clip farm through a month-long season and are scored on their cash at the end.
- Leader
- $9,514
- GPT-6 Astra, mean cash, from $300
- Scored on
- Settled cash
- Field
- 19 models
Full results →
Short Form Bench v1.0
7 days seasonModels run a short-form video brand (account, store, inbox and bank) for a week and are scored on what the business is worth.
- Leader
- $2,949
- GPT-6.1 Sol, business value, from $1,833
- Scored on
- Business value
- Field
- 9 models
Full results →
Agent
Harness studies. The model is held fixed and only the harness changes, so every gap is the harness: tasks solved, and what each successful task cost.
Long Horizon Terminal Bench
August 2026Multi-hour terminal tasks run unattended in real sandboxes — five harnesses across three completion windows, one fixed model throughout.
- Most tasks solved
- 13 / 16 (81.3%)
- Naive Managed Agents · priority
- Cost / successful task
- $0.1709
- Naive Managed Agents · loose
- Field
- 5 harnesses · 3 windows
Full results →
SWE-Bench Pro
September 2026Real repository issues resolved end-to-end — five harnesses on the same fifteen-task slice with GLM-5.3-Flash throughout.
- Most tasks solved
- 15 / 15 (100.0%)
- Claude Code · GLM-5.3-Flash
- Cost / successful task
- $0.0398
- Naive Managed Agents · GLM-5.3-Flash
- Field
- 5 harnesses · 15 tasks
Full results →
Business Bench
September 2026Spreadsheets, bookkeeping, reports, extraction, drafting, and tooling — 25 matched business operations tasks and one fixed model.
- Cost / successful task
- $0.0223
- 13% less than Stock Pi
- Tasks solved
- 12 / 25 (48.0%)
- Naive Managed Agents
- Field
- 5 harnesses · 25 tasks
Full results →
TheAgentCompany
September 2026Workplace tasks spanning code, chat, documents, spreadsheets, and project tools — ten matched tasks and one fixed model.
- Cost / successful task
- $0.0225
- 26% less than Codex
- Tasks solved
- 6 / 10 (60.0%)
- Naive Managed Agents
- Field
- 5 harnesses · 10 tasks
Full results →
PRDBench
September 2026Product requirements implemented and judged criterion by criterion — five matched tasks, one model, and a 45-minute limit.
- Cost / successful task
- $0.0194
- 7% less than Codex
- Tasks solved
- 4 / 5 (80.0%)
- Naive Managed Agents
- Field
- 4 of 5 harnesses · 5 tasks
Full results →
YC-Bench
September 2026A startup run autonomously for a simulated year — hiring, research, payroll, deadlines, and bankruptcy risk across three seeds.
- Cost / successful task
- $0.825
- 29% less than Stock Pi
- Tasks solved
- 3 / 3 (100%)
- Naive Managed Agents
- Field
- 5 harnesses · 3 seeds
Full results →
CoffeeBench
September 2026A roaster operated inside a six-company agent economy — sourcing, negotiation, inventory, sales, and cash discipline.
- Cost / successful task
- $0.337
- Tuned build, different seeds
- Tasks solved
- 5 / 5 (100%)
- Naive Managed Agents
- Field
- 4 harnesses · 5 runs each
Full results →
Further studies publish here once every cell clears the same verification bar.