Pay once for context.

Conversation prefix caching

When you resend the earlier turns of a conversation within an hour, those earlier turns bill at the cached input rate instead of the input rate. Nothing to turn on — keep the earlier messages byte-for-byte identical and in the same order.

• Put stable content (system prompt, documents, tool definitions) first.
• Append new turns at the end; do not edit earlier ones.
• The cached share shows up in usage.prompt_tokens_details.cached_tokens.

We keep hashes of conversation prefixes for one hour, not their content.

Response caching

An exact repeat of a request at temperature: 0 is served from cache for one hour, at cached rates. You can force caching on at any temperature, or bypass it:

Setting the header

curl https://api.failure.fail/v1/chat/completions \
  -H "Authorization: Bearer $FAILURE_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-tempest-cache: off" \
  -d '{"model": "failure-tempest-1", "messages": [{"role": "user", "content": "Give me a fresh idea."}]}'

# Per request
resp = client.chat.completions.create(
    model="failure-tempest-1",
    messages=[{"role": "user", "content": "Summarise RFC 9114 in five lines."}],
    extra_headers={"x-tempest-cache": "on"},
)

# Or for every request from this client
cached = OpenAI(
    base_url="https://api.failure.fail/v1",
    api_key=os.environ["FAILURE_API_KEY"],
    default_headers={"x-tempest-cache": "on"},
)

const client = new OpenAI({
  baseURL: "https://api.failure.fail/v1",
  apiKey: process.env.FAILURE_API_KEY,
  defaultHeaders: { "x-tempest-cache": "on" },
});

// Or per request
await client.chat.completions.create(
  { model: "failure-tempest-1", messages: [{ role: "user", content: "Fresh take, please." }] },
  { headers: { "x-tempest-cache": "off" } },
);

Brief cache

The pre-answer brief for a prompt is cached for 24 hours inside the pipeline. You never pay for briefs, so this affects latency, not your bill.

Seeing cache hits

Check usage.prompt_tokens_details.cached_tokens for prefix hits, and the tempest.cache field to see whether the response cache served the answer.

What we keep

Response cache entries live one hour, briefs 24 hours, prefix hashes one hour. After that they are gone. See security for the full picture.