Docs / Guides
Long streaming
Keepalive comments every 15 seconds, streams of up to 2 hours, and what happens when a stream breaks.
Last updated
When to stream
Requests that can take longer than 100 seconds must use stream: true. Long agent turns, large outputs and slow reasoning models are the usual cases.
How a stream starts
Cent AI sends the 200 status and the text/event-stream headers right away, before the model starts answering. From then on it sends a comment line every 15 seconds:
: keepalive
Lines that start with : are comments in server-sent events. Clients ignore them, but they keep proxies and idle timers from closing a quiet connection, for example while a reasoning model thinks.
Limits
| Limit | Value |
|---|---|
| Keepalive interval | 15 seconds, from the first second |
| Time to the first output from the model | Up to 600 seconds in total across retries; after that the request fails with upstream_timeout |
| Maximum stream duration | 2 hours by default; some models have a different maximum |
If a model fails
- Before the first byte. Cent AI retries the request on another route for the same model. You do not see the failure, and nothing is charged for the failed attempt.
- After the stream started. There is no retry. The stream ends with an error in the format of your protocol, and the tokens already delivered are charged.
data: {"error":{"message":"The model is temporarily unavailable. Try again shortly.","type":"server_error","code":"upstream_unavailable"}}
data: [DONE]
The Responses endpoint uses the response.failed event with the same codes.
If your client disconnects
Closing the connection does not cancel the charge for what the model already produced. The request is recorded with the tokens delivered so far. If the usage did not arrive before the disconnect, the tokens are counted with a tokenizer and the request is marked estimated.
Client timeouts
Many HTTP clients have a total request timeout. Set it longer than your longest expected stream, or turn it off for streaming calls and rely on the read timeout: the keepalive comments arrive every 15 seconds. See Stream timeout.