Prompt caching: Save 20–73% on LLM API Costs
Does prompt caching save money on LLM API costs?
Yes, for the right workload shape: 20–73% based on Net monthly savings across 3 sourced providers, priced models, and cacheable workload shapes: pricingHub input/output rates plus each provider's sourced write/read multipliers and TTL, with one write per TTL window. It does not apply universally — see the applicability and tradeoff sections below before using it.
Batch 40 · server-rendered decision evidence · verified 2026-08-27
Prompt-caching break-even evidence
Frozen inputs, formulas, provenance, and decision boundaries are visible in the initial HTML. Unsupported evidence fails closed as Unavailable.
Prefix mutation canary
Formula / scoring rule: Cache claim = returned cache read/write/miss units + stable prefix hash; visual similarity is not a hit.
Provenance: Frozen byte-identical, whitespace, reordered-tool, dynamic-date, user-specific, and model-changed prompts.
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | State |
|---|---|---|---|---|
byte-identicalbatch40-prompt-caching-m1-r1 | prefix hash p40; 4,096 tokens; same model; second request 30s later | Returned cache status Unavailable — provider response field not captured | No savings until explicit read units are returned. | Unavailable — provider response field not captured |
whitespace-onlybatch40-prompt-caching-m1-r2 | hash changes; 4,098 tokens; same semantic text | Cache boundary Unavailable — not observable from request log | Treat mutation as a miss until provider confirms prefix matching. | Unavailable — not observable from request log |
model-changedbatch40-prompt-caching-m1-r3 | same text; model ID changes; same tenant | Cache reuse Unavailable — model-scoped behavior is undocumented for this fixture | Never transfer a hit across model IDs. | Unavailable — model-scoped behavior is undocumented for this fixture |
Module citation: Anthropic prompt caching documentation.
TTL-and-arrival replay
Formula / scoring rule: Net savings = uncached bill − (write charges + read charges + misses); expiry and minimum-prefix failures are charged.
Provenance: Identical monthly request volume across bursty, steady, diurnal, and daily schedules; provider TTL terms are sourced per row.
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | State |
|---|---|---|---|---|
burstybatch40-prompt-caching-m2-r1 | 10K requests/hour; 4-hour burst; 8K stable prefix | Reuse count and write/read rates Unavailable — provider-specific pricing is not joined | Do not publish a savings percent from request count alone. | Unavailable — provider-specific pricing is not joined |
steadybatch40-prompt-caching-m2-r2 | 240K/day; one request every 6s; documented TTL candidate | Break-even reuse formula = write/(uncached−read); numeric result Unavailable — read/write rates are absent | Formula remains visible; numeric savings fail closed. | Unavailable — read/write rates are absent |
once dailybatch40-prompt-caching-m2-r3 | 30 arrivals/month; 24h gap; 8K prefix | Expiry count Unavailable — eviction/TTL behavior is not observed | Undocumented expiry cannot be treated as zero. | Unavailable — eviction/TTL behavior is not observed |
Module citation: Google Gemini context caching documentation.
Cache rollout reconciliation panel
Formula / scoring rule: Realized saving = online uncached baseline invoice − cache invoice, normalized by accepted request count.
Provenance: Shadow 10%, canary 20%, production 70% rollout; rollback if hit-rate telemetry or answer parity is absent.
| Frozen fixture / run | Visible inputs | Field-level result | Decision boundary | State |
|---|---|---|---|---|
shadowbatch40-prompt-caching-m3-r1 | 10K requests; prefix hashes logged; no billing change | Observed hit rate Unavailable — cache usage telemetry is absent | Shadow similarity is not billing evidence. | Unavailable — cache usage telemetry is absent |
canarybatch40-prompt-caching-m3-r2 | 20K requests; semantic parity sample 200; rollback threshold 2% mismatch | Parity 198/200; realized invoice delta Unavailable — invoice attribution is absent | Do not scale on parity without cost attribution. | Unavailable — invoice attribution is absent |
rollbackbatch40-prompt-caching-m3-r3 | cache boundary changed; mismatch 3/100; production share 70% | Rollback trigger fires; savings after rollback Unavailable — post-rollback invoice not yet available | Rollback is the safety boundary, not a savings claim. | Unavailable — post-rollback invoice not yet available |
Module citation: All AI Ask evidence registry (verified 2026-08-27).
How it works
Prompt caching stores a stable prefix so later requests pay the provider’s cache-read rate instead of the normal input rate. The economics are not a universal 90% rule: this guide joins each provider’s sourced read multiplier, write multiplier, minimum prefix, and TTL to the same model prices used by the cost calculator. A first write can cost more than an ordinary input; the break-even reuse count is therefore calculated as the write premium divided by the per-read saving. Keep stable instructions and reference material first, put changing user content after the cache boundary, and measure cache hits in production.
Where it applies
Stable-prefix workloads: long system instructions, tool definitions, repeated documents, and multi-turn agents. It does nothing when the prefix changes every request, falls below the provider minimum, or is reused too infrequently to repay the first write.
Prompt-cache economics by provider
| Provider | Cache read | Cache write | Minimum prefix | TTL | Break-even reuses | Verified |
|---|---|---|---|---|---|---|
| OpenAI | 10% of input | 1.25× input | 1,024 tokens | 30 min | 1 reuse | 2026-08-15 |
| Anthropic | 10% of input | 1.25× input | 1,024 tokens | 5 min | 1 reuse | 2026-08-15 |
| 10% of input | 1× input | 4,096 tokens | 60 min | First reuse | 2026-08-15 | |
| xAI | — | — | — | — | — | — |
| DeepSeek | — | — | — | — | — | — |
| Mistral | — | — | — | — | — | — |
| Groq | — | — | — | — | — | — |
| Cerebras | — | — | — | — | — | — |
| Qwen | — | — | — | — | — | — |
| Amazon | — | — | — | — | — | — |
| Z.ai | — | — | — | — | — | — |
| Meta | — | — | — | — | — | — |
Rows marked — are providers without a published provider-level cache record; they are excluded from the break-even and savings calculations. Read and write rates are multipliers on the provider’s ordinary input price.
Worked cache example
This llm chatbot example uses the same workload shape as the cost calculator. The break-even count is the first reuse at which the sourced write premium is repaid by cheaper reads; it is not a generic caching rule.
For a 12,000-token stable prefix, the provider rows below show whether caching is free to enable or how many reuses repay the first write. Confirm that your actual prefix clears each provider’s minimum before using the estimate.
The tradeoff
Caching can increase the bill when a prefix is written and then expires before enough requests reuse it. A changing system prompt, reordered tool list, or short prefix causes misses, so the application pays normal input pricing without receiving the expected read discount. Explicit caches can also add storage charges or require cache-control fields, and cached content has a provider-specific lifetime. Treat the result as a workload decision: use the break-even count below, verify that your prefix clears the minimum token length, and monitor hit and write tokens rather than assuming every repeated-looking request is a hit.
Worked example
For llm chatbot (growing conversation history in, short reply out) at 200,000 calls/month, GPT-5 Nano runs $52.00/month at list price and $52.00/month once verbosity is priced in. See the full ranked table for this workload →