A mental model for `yield from`

To grasp the difference between yield and yield from in Python, it’s helpful to start with how each handles the flow of control and values between generators. Essentially, yield is like a "pause button" on a generator that temporarily yields a value back to the caller, while yield from is a more sophisticated delegation mechanism, allowing one generator to delegate part of its operations to another generator. Here's a breakdown of how to understand yield from with a mental model.

yield — A Simple Flow of Values

When you use yield, you are effectively saying, "here's a value from my generator; take it, and I'll pick up where I left off when you ask me for the next one." The yield statement pauses the generator's execution and provides a single value back to the caller, who can then continue asking for more values.

def my_gen():
    yield 1
    yield 2

With this, calling next(my_gen()) will yield 1 first, and then 2 upon a second call.

yield from — Delegating to Another Generator

yield from, on the other hand, can be thought of as creating a "tunnel" between the outer generator and an inner generator or iterable. Instead of having to manually iterate through the inner generator yourself, yield from does the work for you and handles the back-and-forth communication between all parties involved: the outer generator, the inner generator, and the caller.

A More Sophisticated Flow

The mental model here is: imagine yield from as connecting pipes between the caller (who consumes values), the delegating generator (the one with yield from), and the inner generator (the one being delegated to). All values, exceptions, and input values (send, throw, etc.) flow through this tunnel, with yield from managing the flow transparently.

def inner_gen():
    yield 1
    yield 2

def outer_gen():
    yield from inner_gen()
    yield 3

In this example:

• The caller gets 1 and 2 from inner_gen as if they were directly yielded by outer_gen.
• Once inner_gen is exhausted, outer_gen yields 3.

Handling of send() and throw()

The yield from construct also forwards anything sent to the delegating generator to the inner generator, such as values sent with send(), or exceptions thrown with throw(). Here's how it works:

• send(value): The value is forwarded to the inner generator's yield, continuing its execution.
• throw(exception): The exception is thrown into the inner generator, and if not caught there, it propagates back to the outer generator.
• Return values: If the inner generator has a return statement (introduced in PEP 380), yield from will catch that returned value as a special kind of StopIteration.

Mental Model — A Transparent Proxy

A good analogy for yield from is a transparent proxy or "subroutine delegation":

• Imagine that outer_gen says, "I'm going to pause myself and let inner_gen take over completely until it's done."
• When the caller asks outer_gen for a value (using next(), send(), or throw()), it doesn't need to know that inner_gen is actually handling things — yield from makes it look like outer_gen itself is providing those values.

A Simpler Simulation with Loops

A way to "simulate" what yield from does mentally is to think of it as looping over the inner generator manually:

def outer_gen():
    for value in inner_gen():
        yield value
    yield 3

However, this only covers the case of plain iteration, not the forwarding of send() or throw() calls. The yield from essentially hides all the complexities of managing that interaction, making it more than just a simple loop.

Summary of yield from:

• Efficient Delegation: It provides all values from an inner generator transparently.
• Pass-through Mechanism: Any inputs (send, throw) are forwarded directly to the inner generator.
• Automatic Handling of Return Values: Catches the StopIteration exception from the inner generator to access any returned value.

So, the best mental model might be to think of yield from as a direct link or "channel" between the caller and the inner generator, with yield from acting as a transparent pass-through operator that efficiently delegates the entire control to another generator until it’s finished.

Continue this conversation yourself