Streaming And Usage
Use streaming safely with OpenAI-compatible SDKs, include usage in the final stream event, and inspect Aegis routing headers.
Enable Streaming
Set `stream: true` in the request body to receive incremental SSE chunks instead of waiting for the full completion.
Aegis preserves the standard OpenAI-compatible streaming shape, so SDKs that already support chat completion streaming should work without custom parsing.
Python
stream = client.chat.completions.create(
model="glm-5.2",
messages=[{"role": "user", "content": "Write a commit message for this change."}],
stream=True,
)
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)Node.js
const stream = await client.chat.completions.create({
model: "glm-5.2",
messages: [{ role: "user", content: "Write a commit message for this change." }],
stream: true,
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content || "");
}Include Usage In The Stream
If you want token counts at the end of a streamed response, request them explicitly with `stream_options.include_usage`.
stream = client.chat.completions.create(
model="glm-5.2",
messages=[{"role": "user", "content": "Hello!"}],
stream=True,
stream_options={"include_usage": True},
)
usage = None
for chunk in stream:
if chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="")
if chunk.usage:
usage = chunk.usage
print(usage)Raw SSE Clients
If you build on raw `fetch`, `curl -N`, or low-level HTTP streams, parse each `data:` line until `data: [DONE]`.
Some upstream providers may append extra metadata outside the ordinary content chunks. High-level SDKs usually ignore that, so the supported Aegis contract for end users is the streamed content, the optional final usage block, and the dashboard.
Aegis Headers And Diagnostics
Aegis adds routing diagnostics to responses so you can understand how a request was served.
- `X-Aegis-Route`: which upstream path served the request.
- `X-Aegis-Cache`: whether the request hit or missed the cache-aware routing path.
- `X-Aegis-Cache-Refresh`: whether Aegis successfully refreshed the cached session state after the response.