10.05

Streaming an agent into FastAPI

skillmaxingPython with AIStreaming an agent into FastAPI
Visual lesson

Streaming an agent into FastAPI

Included with Pro · 4:49
Lessons · 1,024 words

A support answer that takes four seconds to appear all at once feels broken. The same answer arriving word by word from the first half-second feels alive, even though it finishes at the same moment. That is why Relay's /chat endpoint streams, and it is why lesson 08.04 taught you to stream a single model call. An agent run is harder, because it is several model calls with tool calls and possibly a handoff in between, and the client wants to know about those too: "looking up your account", "transferring you to billing", not just silence and then text.

Runner.run_streamed gives you all of it as one async stream of events. Some events are raw token deltas from the model. Others mark a tool call starting, a tool result arriving, or the active agent changing. Your job in the endpoint is to pick the events the client cares about, turn each into a line of server-sent events, and yield it.

Pro

Continue reading

Pro unlocks every video lesson, the full notes and runnable code across the Python with AI course, from the language itself to agents, MCP, Langfuse and deployment.

Unlock full access to
Python with AIAI researchComplete system design
30% off with LAUNCH30ends in 7h 40m
Buy now
View all plans ·Already Pro? Sign in
124 online