Generators and iterators

Every for loop in Python asks the same two questions. "Give me an iterator" (iter(x)), and then, repeatedly, "give me the next value" (next(it)) until the iterator raises StopIteration. Lists, dicts, files and database cursors all answer those questions, which is why one loop syntax works on all of them. Anything that answers them is an iterable, and you can write your own.

The cheapest way to write one is a generator: a function that contains yield. Calling it runs nothing. It hands back a generator object, and each next() runs the body up to the next yield, pauses there with all its local variables intact, and returns the value. The function's state lives between calls. That pause is the whole trick, and it is why a generator can walk a table of ten million rows while holding one row in memory.