Docs menu

Docs / Guides

Prompt caching

How long conversations keep hitting the model's cache through Cent AI, and how cache tokens are charged.

Last updated


What prompt caching saves

Many models keep a cache of the prompt prefix they have seen. When the next turn of a conversation starts with the same prefix, those input tokens are read from the cache and charged at the cache read rate, which is much lower than the input rate. Coding agents resend the whole conversation on every turn, so caching is where most of their cost goes.

What Cent AI does

  • Same route for a conversation. After the first turn, the next turns of the same conversation go to the route that served the previous turn, so the model's cache can be hit. You do not add any field to your request for this.
  • Conversation identity. Cent AI uses the session signal your tool sends when there is one. Without it, it uses the stable start of the prompt, not the parts that change every turn.
  • Your cache controls are kept. cache_control on /v1/messages and other request fields are passed to the model unchanged. Cent AI adds no session headers of its own.
  • Failover. If the route of a conversation fails, the conversation moves to the next route. The first turn there may miss the cache.

Cent AI does not promise a cache hit rate. Whether a turn hits the cache depends on the model and on how your tool builds the prompt.

How cache tokens are charged

Cache rates apply only when the model reports cache tokens for the request. Cent AI never estimates or assumes cache usage.

FormatCache readCache write
Chat Completionsusage.prompt_tokens_details.cached_tokens—
Responsesusage.input_tokens_details.cached_tokens—
Messagesusage.cache_read_input_tokensusage.cache_creation_input_tokens, with cache_creation.ephemeral_5m_input_tokens and ephemeral_1h_input_tokens when reported

The four rates are explained in Pricing; cache writes are billed at the input rate.

Checking it

Open Requests in the dashboard: each request shows its tokens per rate, so you can see how much of a turn came from the cache.