Timeouts, cancellation, and semaphores

The fan-out from the last lesson has three problems the moment it meets production. A model call can hang for a minute, and nothing bounds it. A client can close the browser tab while two models are still generating, and the work continues at your expense. And a nightly job that re-classifies ten thousand tickets with gather will open ten thousand simultaneous requests, which the provider will refuse and your rate limit will remember.

asyncio has one tool for each. asyncio.timeout bounds a wait. Cancellation, delivered as a CancelledError raised inside the coroutine at its current await, is how a coroutine is told to stop, and TaskGroup uses it to make a group of tasks fail or succeed together. asyncio.Semaphore limits how many coroutines can be inside a block at once. Together they turn a demo into something you can run unattended.