Skip to main content
Several SynapsAI Cloud endpoints support streaming so you can display partial output as the model generates it. Streaming uses Server-Sent Events (SSE) over HTTP, matching the OpenAI streaming format.

Supported endpoints

See Supported tasks for the full list.

Python SDK

Pass stream=True to receive an iterator of chunks:

cURL

Set "stream": true in the request body:
The -N flag disables buffering so chunks arrive as they are sent.

Response format

Each SSE line is a data: event containing a JSON object. A typical chunk looks like:
The stream ends with:

Tool calling with streaming

When using tools, streamed chunks may include delta.tool_calls with partial function arguments. Accumulate tool call fragments across chunks before invoking functions. See Tool calling for a complete example.

Error handling

If an error occurs mid-stream, the connection may close with a non-200 status before [DONE]. Wrap streaming loops in try/except and implement retry logic with exponential backoff for transient 429 or 500 responses. See API overview and Troubleshooting for error details.

Production tips

  • Set reasonable max_tokens to bound latency and cost.
  • Use Production or Serverless deployment for user-facing streaming to avoid first-token delay from instance load time.
  • Implement client-side timeouts and cancellation (close the HTTP connection to stop generation when the user navigates away).