Streaming an agent into FastAPI
Streaming an agent into FastAPI
A support answer that takes four seconds to appear all at once feels broken. The same answer arriving word by word from the first half-second feels alive, even though it finishes at the same moment. That is why Relay's /chat endpoint streams, and it is why lesson 08.04 taught you to stream a single model call. An agent run is harder, because it is several model calls with tool calls and possibly a handoff in between, and the client wants to know about those too: "looking up your account", "transferring you to billing", not just silence and then text.
Runner.run_streamed gives you all of it as one async stream of events. Some events are raw token deltas from the model. Others mark a tool call starting, a tool result arriving, or the active agent changing. Your job in the endpoint is to pick the events the client cares about, turn each into a line of server-sent events, and yield it.
Continue reading
Pro unlocks every video lesson, the full notes and runnable code across the Python with AI course, from the language itself to agents, MCP, Langfuse and deployment.