Prompt caching explained: Anthropic, OpenAI and Gemini compared

Most production prompts repeat themselves. A coding agent resends the same system prompt and tool definitions on every turn. A support bot sends the same policy document with every question. A retrieval pipeline sends the same instructions around different passages. You pay for all of those input tokens, every time.

Prompt caching lets the provider keep its processing of a repeated prompt prefix and bill the repeated part at a reduced cached-input rate. Every major provider offers it. They differ in whether it is automatic, how long entries live, and how writes and reads are billed, and those differences decide whether you actually get the discount.

The rule every provider shares: the prefix must be identical

A cache hit needs the beginning of the prompt to match an earlier request exactly, token for token, up to the point being reused. Anthropic’s documentation puts it plainly: a cache hit requires the prompt to be completely identical up to and including the cached block. OpenAI’s says the entire rendered prefix must match, and that a change before a breakpoint means everything after it cannot match.

That has a practical consequence for how you build prompts:

1. Put stable content first. Tool definitions, the system prompt, long reference documents and few-shot examples go at the start.
2. Put variable content last. The user’s question, retrieved passages that change per request, timestamps and user IDs go at the end.
3. Do not inject anything dynamic into the stable part. A current date or request ID in the system prompt silently breaks every cache hit after it.
4. Keep tool definitions in a fixed order. A tool list rebuilt in a different order each time is a different prefix.

The order also matters across sections. On Anthropic the cached prefix is built from tools, then system, then messages, so a change to the tools invalidates everything after them.

Anthropic (Claude)

• Opt-in. Caching is switched on with cache_control, either once at the top level of the request (the cache point then moves forward automatically as a conversation grows) or on individual content blocks for precise control. A request without it is not cached.
• Minimum length. Each model has a minimum cacheable prefix, from a few hundred to a few thousand tokens. A shorter prompt marked for caching is simply processed without caching, with no error.
• Lifetime. Entries live five minutes by default and the timer resets each time the entry is used. A one-hour lifetime is available at a higher write price.
• Billing. Writing a cache entry costs more than ordinary input, and reading it costs far less. The one-hour lifetime costs more to write than the five-minute one. The exact multipliers, which vary by model, are on Anthropic’s pricing page.
• The same mechanism applies to Claude on AWS Bedrock and Google Vertex AI.

Because a write costs more than a plain request, caching on Anthropic pays off only when the prefix is reused before it expires. A prefix written once and never read again costs more than not caching at all.

OpenAI

• Automatic. Prompt caching is on by default for supported models; there is nothing to switch on. The newest models also accept explicit cache breakpoints.
• Minimum length. Prompts below roughly a thousand tokens are not cached.
• Routing hint. Cached state lives on individual machines. On models before GPT-5.6, setting prompt_cache_key to the same value for requests that share a prefix helps them land where the cache is; from GPT-5.6 OpenAI routes for the cache itself.
• Lifetime. Depends on the model: older models offer short in-memory retention or extended retention of up to a day; the newest models let you request a minimum lifetime.
• Billing. Reused tokens are billed at the model’s cached-input rate. On the newest models a cache write carries a premium over ordinary input, and a longer lifetime costs more to write. Details are in OpenAI’s prompt caching guide and pricing page.

Google Gemini

Gemini offers two kinds of caching:

• Implicit caching is on by default for Gemini 2.5 and newer models. Google passes on the saving automatically when a request hits a cache, but does not guarantee hits.
• Explicit caching lets you create a named cache of content and reference it in later requests. Its time-to-live defaults to one hour, and you pay for storage for as long as the cache exists, on top of the reduced rate for cached tokens. On the Gemini API it is in beta and is not supported from Gemini 4 Argon onward, and Vertex AI’s explicit-caching model list does not include Gemini 4 Argon either; those models rely on implicit caching.

Google’s advice for implicit caching matches everyone else’s: put large, common content at the beginning of the prompt, and send requests with a similar prefix close together in time. See the Gemini API caching guide and the Vertex AI context cache overview.

Why caching often fails in practice

Teams switch caching on and see little change on the invoice. The usual causes:

• The prefix is not actually stable. A timestamp, a session ID, or a reordered tool list in the first part of the prompt.
• The cache expires between requests. Traffic for one prefix arrives less often than the entry’s lifetime, so every request pays to write and none read.
• Requests land on different machines. On providers that cache per machine, related requests without a shared routing key miss each other’s entries.
• Nobody added the markers. On Anthropic, an application that never sets cache_control gets no caching, however repetitive its prompts are.
• Nobody measures it. Every major provider reports cached-input tokens in the response’s usage block, and Anthropic and OpenAI report cache writes too. If you are not recording them, you cannot tell whether caching is working.

How a gateway helps

Getting this right in every application, for every provider, is tedious, and it drifts as prompts change. A gateway can do it once, for every request.

The Tessarac AI Cost Optimizer adds what each provider needs on the way through: cache_control breakpoints at the end of the tool definitions, the system prompt and the conversation for Claude (never more than Anthropic allows, and never changing breakpoints you set yourself), a prompt_cache_key derived from the system prompt and tools for OpenAI and Azure OpenAI, and a reference to an existing cached context for Gemini on Vertex. It also lets you choose the cache lifetime per request, per key or per connection. The provider computes the same answer either way: only the billing changes. See Caching in the documentation.

Prompt caching reduces the cost of repeated input. To avoid the provider call entirely for repeated requests, see semantic caching, and for the full list of techniques, how to reduce LLM API costs.