Every time you use an AI through an API, you pay for two kinds of tokens: input tokens (what you send in) and output tokens (what the model writes back). Output tokens cost five to ten times more — not arbitrarily, but because generating each one requires a full forward pass through the model, sequentially, one word at a time. Input tokens can be processed in parallel, making them far cheaper to handle.

Prompt caching exploits this asymmetry on the input side: if you repeatedly send the same large context — a codebase, a policy document, a long system prompt — the provider stores it and charges a fraction of the normal input price on subsequent reads. Anthropic's Claude cut its cache-read price 75% to $0.25 per million tokens, just 2.5% of its standard input price. For AI agents that loop through the same context hundreds of times, that single change can cut total bills by nearly half.