Receive a model’s response as it is generated instead of waiting for the entire answer.

Streaming displays the model’s response progressively as the model generates it, allowing you to start reading before the complete answer is ready. Without streaming enabled, you see nothing while the model generates its answer. After generation finishes, the complete answer appears at once. Both approaches can generate the same answer—the key difference is in how the response is delivered to your application and displayed. Streaming is useful in cases where users benefit from seeing the response immediately.

Enabling streaming

Deploy supports streaming responses via Server-Sent Events (SSE). By default, stream: false. Set stream: true in your request to enable it.

from openai import OpenAI

client = OpenAI(
    base_url="https://api-cdn.deploy.com/api/v1",
    api_key="DEPLOY_API_KEY",
)

stream = client.chat.completions.create(
    model="DEPLOY_MODEL_ID",
    messages=[
        {
            "role": "user",
            "content": "Explain TCP and UDP in two sentences."
        }
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

    if chunk.usage:
        print(f"\n\n[usage] {chunk.usage}")

curl https://api-cdn.deploy.com/api/v1/chat/completions \
  -H "Authorization: Bearer $DEPLOY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "model": "DEPLOY_MODEL_ID",
    "messages": [
      {
        "role": "user",
        "content": "Explain TCP and UDP in two sentences."
      }
    ],
    "stream": true
  }'

Response format

The response is a stream of Server-Sent Events, each an OpenAI-style chat.completion.chunk. Content arrives incrementally in choices[0].delta.content:

data: {"id":"...","object":"chat.completion.chunk","model":"<MODEL_KEY>","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]}

data: {"id":"...","object":"chat.completion.chunk","choices":[{"index":0,"delta":{"content":" there"},"finish_reason":null}]}

The stream includes a final usage chunk with an empty choices array and populated usage field. This is followed by data: [DONE] , marking the end of the stream.

data: {"id":"...","object":"chat.completion.chunk","choices":[],"usage":{"prompt_tokens":18,"completion_tokens":42,"total_tokens":60}}

data: [DONE]

Get Started

Chat Completions

Large Language Models

Security & Compliance

Streaming

Enabling streaming

Response format

​Enabling streaming

​Response format

Enabling streaming

Response format