Prompt Caching in 2026: The Cost and Latency Guide

Step 1 — Anthropic: mark the stable prefix

Add cache_control to the blocks that rarely change. Keep the fresh user message unmarked, and reuse the exact same prefix on the next call. Watch usage for cache_creation_input_tokens on the first call and cache_read_input_tokens afterwards.

import anthropic

client = anthropic.Anthropic()

response = client.messages.create(
    model=“claude-sonnet-4-5-20250929”,
    max_tokens=1024,
    system=[
        {
            “type”: “text”,
            “text”: SYSTEM_PROMPT,  # long, stable instructions
            “cache_control”: {“type”: “ephemeral”},
        }
    ],
    messages=[{“role”: “user”, “content”: user_question}],
)

print(response.usage.cache_creation_input_tokens)
print(response.usage.cache_read_input_tokens)

Step 2 — Anthropic: pick the TTL deliberately

Default TTL is 5 minutes. For agentic loops or long conversations where follow-ups can arrive later, set ttl to “1h” — it doubles the write price but cache reads stay at 0.1x. Mix both TTLs only with 1-hour entries before 5-minute ones.

system=[
    {
        “type”: “text”,
        “text”: SYSTEM_PROMPT,
        “cache_control”: {“type”: “ephemeral”, “ttl”: “1h”},
    }
]

# Or automatic caching: one top-level marker,
# the breakpoint follows the last cacheable block.
response = client.messages.create(
    model=“claude-sonnet-4-5-20250929”,
    max_tokens=1024,
    cache_control={“type”: “ephemeral”},
    system=[{“type”: “text”, “text”: SYSTEM_PROMPT}],
    messages=[{“role”: “user”, “content”: user_question}],
)

Step 3 — OpenAI: choose a mode, place a breakpoint

On GPT-5.6 and later, keep the default implicit mode for append-only threads, or set prompt_cache_options.mode to “explicit” and mark the end of stable content with prompt_cache_breakpoint so the changing suffix is never written at 1.25x. Reuse a stable prompt_cache_key, keep the prefix ≥ 1,024 tokens with TTL 30m (default, only value), then read cached_tokens and cache_write_tokens. Pre-5.6: automatic only, 2,048-token minimum, free writes, prompt_cache_retention instead of prompt_cache_options.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model=“gpt-5.6”,
    prompt_cache_key=“my-app-v1:session-42”,
    prompt_cache_options={“mode”: “explicit”, “ttl”: “30m”},
    input=[
        {
            “role”: “developer”,
            “content”: [
                {
                    “type”: “input_text”,
                    “text”: LONG_SYSTEM_PROMPT,  # stable, >= 1,024 tokens
                    “prompt_cache_breakpoint”: {“mode”: “explicit”},
                }
            ],
        },
        {“role”: “user”, “content”: user_question},  # dynamic last
    ],
)

print(response.usage.input_tokens_details.cached_tokens)
print(response.usage.input_tokens_details.cache_write_tokens)
# Pre-5.6: automatic only, 2,048-token min, free writes,
# prompt_cache_retention (in_memory / 24h), 128-token rounding.

Step 4 — Gemini: rely on implicit, graduate to explicit

On Gemini 2.5+ do the same stable-first ordering and implicit caching just works — check usageMetadata for cached tokens. When you need guaranteed reuse of a big corpus (docs, repo, video), create an explicit cache with a TTL and reference its name on later calls.

from google import genai

client = genai.Client()

cache = client.caches.create(
    model=“gemini-2.5-flash”,
    config={
        “system_instruction”: LONG_SYSTEM_PROMPT,
        “contents”: [LARGE_DOCUMENT_CORPUS],
        “ttl”: “3600s”,
    },
)

response = client.models.generate_content(
    model=“gemini-2.5-flash”,
    contents=f“Question: {user_question}”,
    config={“cached_content”: cache.name},
)

print(response.usage_metadata)

1. Dynamic content before stable content

A timestamp, user ID, or changing RAG snippet at the top of the prompt changes the prefix hash, so every request is a miss. Fix: move anything variable after the cached blocks.

2. Prefix below the minimum, marked anyway

Anthropic and Gemini ignore cache marks on prefixes under the model’s minimum (1,024–4,096 tokens depending on model). Short prompts cache nothing — no error, just no savings. Fix: only mark blocks you know clear the threshold; verify with the token counter.

3. Tool schemas serialized in random order

Dict iteration or regenerated JSON with shuffled keys produces a byte-different prefix each call. Fix: serialize tools deterministically (sorted keys, fixed order) so the prefix matches byte-for-byte.

4. TTL shorter than your conversation gaps

A 5-minute TTL on a support bot where users reply after 20 minutes means every turn is a fresh write. Fix: measure your inter-request gaps and use the 1-hour TTL (Claude) or explicit cache with a longer TTL (Gemini) when gaps exceed minutes.

5. Growing conversations outrun the lookback window

On Claude, a single breakpoint can fall outside the 20-block lookback as history grows, silently ending hits mid-session. Fix: add a second breakpoint closer to the growing tail — one for stable instructions, one after recent history.