Streaming (SSE)

All streaming endpoints follow OpenAI-style Server-Sent Events.

Enable

In the request body:

{ "stream": true }

Response format

Content-Type: text/event-stream. Each event:

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"An"},"index":0,"finish_reason":null}]}

data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":" old"},"index":0,"finish_reason":null}]}

...

data: [DONE]

Each data: line is a JSON chunk; ends with data: [DONE].

Assemble the message

Concatenate choices[0].delta.content from all chunks in order — that's the full reply.

Tool calls and reasoning are also delta-streamed via delta.tool_calls / delta.reasoning_content.

Get usage

By default chunks don't include usage. To see usage in the final chunk:

{
  "stream": true,
  "stream_options": { "include_usage": true }
}

The last chunk:

data: {"id":"...","choices":[],"usage":{"prompt_tokens":10,"completion_tokens":42,"total_tokens":52}}

Error handling

If something fails mid-stream, a data: chunk with an error field is emitted, then the connection closes:

data: {"error":{"type":"upstream_error","message":"...","code":"upstream_500"}}

Listen and surface gracefully in your client.

SDK examples

Python (openai):

stream = client.chat.completions.create(
    model="qwen-max",
    messages=[...],
    stream=True,
    stream_options={"include_usage": True},
)
for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
    if chunk.usage:
        print(f"\nusage: {chunk.usage}")

TypeScript:

const stream = await client.chat.completions.create({
  model: 'qwen-max',
  messages: [...],
  stream: true,
  stream_options: { include_usage: true },
})

for await (const chunk of stream) {
  if (chunk.choices[0]?.delta?.content) {
    process.stdout.write(chunk.choices[0].delta.content)
  }
  if (chunk.usage) console.log('\nusage:', chunk.usage)
}

Gotchas

• Proxy / CDN buffering: when self-proxying, set X-Accel-Buffering: no and proxy_buffering off. Otherwise streaming feels like "wait forever, then everything at once."
• Timeout: client timeout for streaming should be 10+ minutes.
• Cancellation: client disconnect — upstream often continues to completion. To save money, explicitly abort the request and close the socket.