What Prompt Caching Actually Is

Every time you send a request to an LLM, the model has to process every token in your prompt from scratch. For most applications, a huge chunk of that prompt is identical across requests: the system message, tool definitions, a long document you're analyzing, retrieved context from a RAG pipeline. You're paying full price to recompute something the model just finished computing two seconds ago.

Prompt caching solves this by reusing the intermediate computations from a previous request rather than redoing the work. Under the hood, what's being cached is the KV (key-value) cache from the attention mechanism. When a transformer processes tokens, it generates a key and value matrix for each layer, and those matrices are exactly what gets reused when the prompt prefix matches a cached entry. The model picks up mid-computation instead of starting over.

The savings are real. On current OpenAI and Anthropic models, cached input tokens cost 10% of the regular input price or less (5% on Claude Opus 5.5, Claude Sonnet 5.5 and gpt-6.1-sol, and 2.5% on Claude Fable 5.1). On latency, AWS reports up to 85% reduction in response time when the cache hits. For an AI agent making dozens of tool calls per session, each one re-sending a 4,000-token system prompt, the math gets compelling fast.

How Each Provider Implements It

Anthropic (Claude)

Anthropic's implementation uses explicit cache control markers. You add "cache_control": {"type": "ephemeral"} to the content blocks you want cached. The minimum cacheable prompt is 512 tokens on current models (Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5). Some older models need 1,024 to 4,096 tokens. You can define up to four cache breakpoints per request. Writes to the default 5-minute cache cost 1.25× the base input price, and writes to the optional 1-hour cache cost 2×. Cache reads cost 0.1× base input on most models, 0.05× on Claude Opus 5.5 and Sonnet 5.5, and 0.025× on Claude Fable 5.1.

Anthropic also offers automatic caching: add a single cache_control field at the top level of the request and the API places the cache breakpoint on the last cacheable block, moving it forward as the conversation grows. You can still place explicit breakpoints on individual blocks for finer control. This helps significantly for teams already in production who haven't restructured their prompts for caching. Official documentation: Anthropic Prompt Caching Docs.

OpenAI

OpenAI's approach is automatic and transparent. Caching is on by default. On GPT-5.6 and later, prefixes of at least 1,024 visible tokens can be cached. Cache reads cost 0.1× the input price (0.05× on gpt-6.1-sol), and cache writes cost 1.25×. On earlier models there is no write charge, the minimum varies by request settings, and the cached-input discount depends on the model (for example 90% on gpt-5.4 and 50% on gpt-4o). By default OpenAI picks the breakpoint for you. On GPT-5.6 and later you can also mark explicit breakpoints and prewarm the cache before the first user request. You can see the savings reflected in the usage object in the API response under cached_tokens.

The tradeoff is less control. You still can't clear the cache manually or inspect what's cached, and the cache is per-model, per-organization. On GPT-5.6 and later, a cached prefix stays available for at least 30 minutes after its last write or reuse. Older models keep in-memory entries for about 5 to 10 minutes of inactivity (up to an hour), or up to 24 hours with extended retention. For long-running agent sessions, a cache that was warm at the start of a session may be cold by turn 20. Details: OpenAI Prompt Caching Guide.

Google Gemini

Gemini's implementation, called Context Caching, works differently. You explicitly create a cached content object via the API with a specified time-to-live, then reference that cache in subsequent requests. Gemini 2.5 and newer also have implicit caching, which is on by default with no storage fee but no guaranteed saving. Explicit caching is in beta and works through the generateContent API, not the Interactions API. Its TTL defaults to 1 hour if you don't set one. The minimum is 4,096 tokens on Gemini 3.8 Flash, 3.6 Flash and 3.1 Pro Preview, and 2,048 tokens on Gemini 2.5 Flash and 2.5 Pro. You pay a per-hour storage cost for the cached content, plus a reduced per-token rate when it's used.

This model works well for analyzing a large codebase across many requests, processing a long legal document in multiple passes, or running repeated queries against a fixed dataset. Explicit caching suits large, stable context. Long system prompts over the 4,096-token minimum can benefit too, especially through implicit caching. Google's documentation: Gemini Context Caching.

Prompt caching cost calculator

Updated 12 Oct 2026

Use this calculator to see what prompt caching would save on your own workload. Enter the size of the prefix that stays the same on every request (system prompt, tool definitions, reference documents), the tokens that change per request, and your traffic. It compares cost with and without caching on current Anthropic, OpenAI and Google Gemini models, using prices from each provider's official pricing page.

Prompt caching prices by model (verified 12 Oct 2026)

Standard API prices in US dollars per million tokens, taken from each provider's official pricing page on 12 Oct 2026. Batch, priority/fast and data-residency pricing are not included.

Anthropic (pricing). Minimum cacheable prompt: 512 tokens.

• Claude Fable 5.1: $10 input, $12.50 5-minute cache write, $20 1-hour cache write, $0.25 cache read, $50 output.

• Claude Opus 5.5: $4 input, $5 5-minute cache write, $8 1-hour cache write, $0.20 cache read, $20 output.

• Claude Sonnet 5.5: $2 input, $2.50 5-minute cache write, $4 1-hour cache write, $0.10 cache read, $10 output.

• Claude Haiku 5.5 (prompts up to 100,000 tokens): $0.10 input, $0.125 5-minute cache write, $0.20 1-hour cache write, $0.01 cache read, $0.50 output.

• Claude Haiku 5.5 (prompts over 100,000 tokens): $0.50 input, $0.625 5-minute cache write, $1 1-hour cache write, $0.05 cache read, $2.50 output.

OpenAI (pricing), prompts up to 272K tokens. Minimum cacheable prefix: 1,024 tokens.

• gpt-6-astra: $10 input, $12.50 cache write, $1.00 cache read, $50 output.

• gpt-6.1-sol: $2 input, $2.50 cache write, $0.10 cache read, $10 output.

• gpt-6-luna: $0.10 input, $0.125 cache write, $0.01 cache read, $0.50 output.

Google Gemini (pricing). No cache write fee; explicit caching adds hourly storage.

• Gemini 3.8 Flash: $0.75 input, $0.075 cached, $0.50 storage per 1M tokens per hour, $3.75 output. Minimum 4,096 tokens. From 1 Jan 2027 Google lists $1.50 input, $0.15 cached, $1.00 storage and $7.50 output.

• Gemini 3.1 Pro Preview (prompts up to 200k tokens): $2 input, $0.20 cached, $4.50 storage per 1M tokens per hour, $12 output. Minimum 4,096 tokens.

• Gemini 3.5 Flash-Lite: $0.30 input, $0.03 cached, $1.00 storage per 1M tokens per hour, $2.50 output.

Longer prompts cost more: OpenAI prompts over 272K tokens and Gemini 3.1 Pro prompts over 200k tokens use higher prices listed on the same pages. Sources: Anthropic prompt caching, OpenAI prompt caching, Gemini context caching.

How we calculate this

• Without caching: every request pays the standard input price for the whole prompt (cached prefix plus per-request tokens) plus the output price.

• With caching: a request that hits the cache pays the cache-read price for the prefix. A request that misses pays the cache-write price: 1.25× input for Anthropic's 5-minute cache, 2× for its 1-hour cache, and 1.25× for OpenAI GPT-5.6 and later. On Gemini, a miss pays the standard input price. Per-request tokens and output tokens always pay full price.

• Hit rate: you can enter your own, or estimate it from traffic and cache lifetime assuming requests arrive at random during active hours: hit rate = 1 − e^(−requests per minute × TTL in minutes). The TTLs used are 5 minutes or 1 hour for Anthropic (each hit refreshes the TTL) and 30 minutes for OpenAI (its stated minimum). Google doesn't publish an implicit-cache lifetime, so for Gemini implicit caching you enter the hit rate yourself.

• Gemini: explicit caching adds storage (cached tokens × hourly storage price × hours stored per day × 30). Implicit caching has no storage fee. Google says implicit savings are passed on automatically but gives no separate rate, so we apply the listed context-caching price to implicit hits.

• Assumptions: a 30-day month, standard-tier USD prices (no batch, priority or data-residency pricing), and the provider's higher long-prompt prices once cached plus per-request tokens pass its threshold. A prefix below the model's minimum cacheable size is not cached. "Break-even" is the number of cache reads needed to recover one write's extra cost. For Gemini explicit caching, it's the reads per stored hour needed to cover storage.

Why Your Cache Hit Rate Is Probably Zero

The most common failure mode is prompt structure. For caching to work, the prefix of your prompt must be identical to a previously cached prefix. That sounds obvious, but a lot of real application code defeats this without anyone realizing it.

Consider a system prompt that includes the current date and time. You're probably doing this so the model knows when today is. Every request now has a unique prefix, and your cache hit rate is exactly 0%. Same problem if you inject the user's name, a session ID, or any dynamic value into the system prompt or early in the context. Anything that varies between requests will break cache reuse for everything that follows it in the prompt.

The fix is to structure your prompts so all static content comes first, and all dynamic content comes last. Static system instructions, tool definitions, and large reference documents should occupy the beginning of the context. User-specific details, conversation history, and the actual user message go at the end. This is the opposite of how many prompts are intuitively written, and changing it is often the single highest-leverage optimization available.

A second common issue appears in agents. Many agent frameworks naively append every tool call result to the conversation, letting context grow unbounded. After several turns, the prefix is no longer consistent across requests because the accumulated history keeps changing. Strategies like summarizing older turns or using a fixed-size sliding window over history preserve the cacheable prefix.

Practical Patterns That Work

RAG with a Stable Document Corpus

If you retrieve the same documents repeatedly across requests (internal documentation, product specs, a reference knowledge base), those documents are excellent cache candidates. Place them in a fixed position in the prompt, before any user-specific content. For Anthropic, wrap the document block with a cache control marker. For OpenAI, ensure the documents always appear at the same position so the prefix stays consistent.

Retrieval order matters here. If your RAG pipeline retrieves documents in different orders depending on the query, the prefix changes and you lose the cache. Consider sorting retrieved documents by a stable key (document ID, for example) before assembling the prompt. This one change can dramatically improve cache hit rates in production RAG systems.

Tool Definitions in Agentic Systems

Tool definitions are high-value caching targets. A moderately complex AI agent might define 15 to 20 tools, which can easily consume 2,000 to 4,000 tokens. Since tool definitions almost never change between requests in a session, they should be positioned early in the prompt and marked for caching.

In multi-agent setups where agents share a common tool registry, caching the tool definitions once and reusing across all agent calls compounds the savings. In a system making 50 tool calls per user session with a 3,000-token tool definition block, caching alone could reduce input costs by 30% or more on that component.

This is one of the first optimizations applied when deploying agents in production at Genta, especially in high-frequency agentic loops where LLM calls repeat with largely identical context on every turn.

Long System Prompts and Personas

Customer-facing AI applications often have extensive system prompts: brand voice guidelines, compliance instructions, domain-specific rules, worked examples. These can run to several thousand tokens. If the system prompt is the same across all users, it's a perfect cache candidate.

The failure mode here is personalization. If you add user-specific context to the system prompt, you've broken the cache for every user. Move personalization to a user message or a late-stage turn in the conversation instead.

Cache Lifetimes and What Expires When

The documentation doesn't stress this enough: caches expire. Anthropic's cache lasts 5 minutes by default and is refreshed at no cost each time it's read. An optional 1-hour TTL costs 2× base input to write. The lifetime is counted from the start of the request, so a long-streaming response uses part of it. OpenAI's GPT-5.6 and later keep cached prefixes for at least 30 minutes after last use. Older OpenAI models are shorter unless you enable 24-hour retention. Google's context cache is more explicit because you set the TTL yourself and pay for storage time.

For latency-sensitive applications, the first request in a cold session always pays full price. Some teams run lightweight warming requests at session start if they know a user is about to begin an expensive multi-turn interaction. Others run periodic background requests against their most common prompt configurations to keep caches warm between real user sessions.

Cache invalidation also happens automatically when the model version changes. Anthropic and OpenAI both note that prompt caches are model-specific. Any model version upgrade clears the cache, so plan for a temporary cost spike whenever you upgrade in production.

How to Tell If It's Actually Working

OpenAI surfaces cache hits directly in the API response. In the usage object, look for prompt_tokens_details.cached_tokens. A value close to your full prompt token count means the cache is working. A value of zero on repeated, identical requests means it is not, and the prompt structure needs investigation.

Anthropic's response includes a cache_read_input_tokens field. If you're sending requests that should be hitting the cache and this field is consistently zero, check: is your prompt prefix changing between requests? Are you below your model's minimum cacheable length (512 tokens on current Claude models)? Is there a timing gap between requests that causes cache expiry?

Track your average cached token ratio over time as a production metric. Even a 40% cache hit rate on input tokens significantly changes the cost profile of a high-volume LLM application. Teams that haven't measured this often discover that structural changes to their prompts would have saved meaningful money from the start.

What Prompt Caching Cannot Do

Prompt caching only covers input tokens. Output tokens are always billed at full price, so applications that generate long outputs won't see the same cost leverage. The economics are strongest when input tokens dominate output tokens, which is common in analysis, classification, and structured extraction tasks.

Caching also doesn't help with the first request in any session, or after cache expiry. For very low-traffic applications making only a few requests per hour, the cache may never be warm when you need it. The strategy matters most for applications with consistent, repetitive access patterns.

Finally, caching is not semantic. It's byte-for-byte prefix matching. A single character difference in the cached prefix results in a cache miss. You cannot cache similar prompts, only identical prefixes. Templating systems and prompt construction utilities need to produce deterministic output for caching to be reliable in production. Whitespace, token ordering, field ordering in JSON tool schemas: all of it counts.

For teams building production AI systems, combining prompt caching with other cost control strategies (model routing, structured output parsing, batching) is where the real leverage is. These optimizations compound. A system with good cache utilization, a sensible model routing policy, and efficient context management can cost 60 to 70% less than a naive implementation using the same underlying models. That's before any change to what the system actually does. If you want to see how this fits into a broader production architecture, the post on context engineering for AI agents covers the surrounding patterns in depth.

Prompt caching FAQ

How much do Anthropic prompt caching reads and writes cost?

On the Claude API, writing a prefix to the default 5-minute cache costs 1.25 times the model's base input price. Writing to the 1-hour cache costs 2 times. Reading from the cache costs 0.1 times the base input price on most models, 0.05 times on Claude Opus 5.5 and Claude Sonnet 5.5, and 0.025 times on Claude Fable 5.1. For Claude Sonnet 5.5 that works out to $2 per million input tokens, $2.50 for 5-minute writes, $4 for 1-hour writes, $0.10 for cache reads and $10 for output. Output tokens are never discounted. The caching multipliers stack with the Batch API discount and data-residency pricing. Source: Anthropic pricing.

What read discount do Anthropic's prompt caching docs give, and when does a cache write pay off?

Anthropic's prompt caching docs price a cache hit at 10% of the standard input price, a 90% discount. On Claude Opus 5.5 and Claude Sonnet 5.5 a hit costs 5%, and on Claude Fable 5.1 it costs 2.5%. At the standard rate, a 5-minute cache write pays for itself after one cache read, and a 1-hour write after two. Each hit also refreshes the cache lifetime at no extra cost. On current models (Fable 5.1, Opus 5.5, Sonnet 5.5, Haiku 5.5) the prefix must be at least 512 tokens. Shorter prompts are processed without caching and return no error.