Docs / Guides
Prompt caching
How long conversations keep hitting the model's cache through Cent AI, and how cache tokens are charged.
Last updated
What prompt caching saves
Many models keep a cache of the prompt prefix they have seen. When the next turn of a conversation starts with the same prefix, those input tokens are read from the cache and charged at the cache read rate, which is much lower than the input rate. Coding agents resend the whole conversation on every turn, so caching is where most of their cost goes.
What Cent AI does
- Same route for a conversation. After the first turn, the next turns of the same conversation go to the route that served the previous turn, so the model's cache can be hit. You do not add any field to your request for this.
- Conversation identity. Cent AI uses the session signal your tool sends when there is one. Without it, it uses the stable start of the prompt, not the parts that change every turn.
- Your cache controls are kept.
cache_controlon/v1/messagesand other request fields are passed to the model unchanged. Cent AI adds no session headers of its own. - Failover. If the route of a conversation fails, the conversation moves to the next route. The first turn there may miss the cache.
Cent AI does not promise a cache hit rate. Whether a turn hits the cache depends on the model and on how your tool builds the prompt.
How cache tokens are charged
Cache rates apply only when the model reports cache tokens for the request. Cent AI never estimates or assumes cache usage.
| Format | Cache read | Cache write |
|---|---|---|
| Chat Completions | usage.prompt_tokens_details.cached_tokens | — |
| Responses | usage.input_tokens_details.cached_tokens | — |
| Messages | usage.cache_read_input_tokens | usage.cache_creation_input_tokens, with cache_creation.ephemeral_5m_input_tokens and ephemeral_1h_input_tokens when reported |
The four rates are explained in Pricing; cache writes are billed at the input rate.
Checking it
Open Requests in the dashboard: each request shows its tokens per rate, so you can see how much of a turn came from the cache.