Models
Failure Tempest 1All modelsStatus
Research
OverviewAPI fusionEvaluation
Developers
DocumentationConsoleChangelog
Company
PricingEnterpriseAboutSecurityContact
Account
Sign inCreate account
Home/Research/Evaluation
Evaluation

Measured, or blank.

We have not published a single benchmark number yet, because we have not run one on a live panel yet. This page sets out how we will measure, what we are aiming for, and the rule that losses are published alongside wins.

The goal

Target #1 on Artificial Analysis.

That is the target for Failure Tempest 1. It is not a result. Until an independent run places it, the leaderboard below shows our row as a target and every score as "—".

#ModelScore
—Failure Tempest 1 · target #1—
—Claude Opus 5.5—
—Claude Fable 5.5—
—GPT-6 Astra 6.2—
—Gemini 4 Argon—
Rules

Four rules we keep.

Our own runs

No number is quoted unless it came from a run we did, or an independent leaderboard's run of our model.

Losses published

Where fusion does worse than a single panel model, the loss sits next to the wins.

Same harness

Every model, ours and the panel's, runs through the same prompts, settings and grader.

Cost and time

A fused answer costs more and takes longer. Both are reported beside quality.

Planned table

What we will report.

Each Tempest tier against each panel model on its own. Every cell stays blank until a run fills it.

ModelReasoningCodingMathsKnowledgeLatencyCost / task
Failure Tempest 1 Maxfailure-tempest-1-max——————
Failure Tempest 1failure-tempest-1——————
Failure Tempest 1 Flashfailure-tempest-1-flash——————
Claude Fable 5.5panel model, alone——————
Claude Opus 5.5panel model, alone——————
GPT-6 Astra 6.2panel model, alone——————
Gemini 4 Argonpanel model, alone——————

Methodology

Compare against the best single model

The honest bar for a fused model is not the average of its panel. It is the best panel model on each task. If Tempest does not beat that, fusion added cost without adding quality on that task, and the table will show it.

Ablate the pipeline

Each stage — brief, lenses, fusion, audit — earns its place only if removing it makes answers worse. We plan to run the pipeline with each stage switched off and report what each one contributes, including any stage that turns out to contribute nothing.

Routing has to be right

failure-tempest-1 sends easy prompts to one model. A router that sends a hard prompt down the fast path costs quality; one that fuses an easy prompt costs time and money. We will report how often each happens.

Research tracks

Expert transplant and weight merging are measured against controls and against their parents: a transplant against a matched control arm, a weight merge against each parent and against a plain average. See expert transplant and weight merging.

Results land here first.

When a run completes, the cell fills, the date goes in the changelog, and the losses come with it.