What's the Purpose of "Send" Function on Python Generators

What is the purpose of the send function on Python generators?

It's used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:

>>> def double_inputs():
...     while True:
...         x = yield
...         yield x * 2
...
>>> gen = double_inputs()
>>> next(gen)       # run up to the first yield
>>> gen.send(10)    # goes into 'x' variable
20
>>> next(gen)       # run up to the next yield
>>> gen.send(6)     # goes into 'x' again
12
>>> next(gen)       # run up to the next yield
>>> gen.send(94.3)  # goes into 'x' again
188.5999999999999

You can't do this just with yield.

As to why it's useful, one of the best use cases I've seen is Twisted's @defer.inlineCallbacks. Essentially it allows you to write a function like this:

@defer.inlineCallbacks
def doStuff():
    result = yield takesTwoSeconds()
    nextResult = yield takesTenSeconds(result * 10)
    defer.returnValue(nextResult / 10)

What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:

def doStuff():
    returnDeferred = defer.Deferred()
    def gotNextResult(nextResult):
        returnDeferred.callback(nextResult / 10)
    def gotResult(result):
        takesTenSeconds(result * 10).addCallback(gotNextResult)
    takesTwoSeconds().addCallback(gotResult)
    return returnDeferred

It's a lot more convoluted and unwieldy.

python3 send() function in generators

I think I've figured it out.

c = gen()

You create a genertor in the c variable, there none of the generator's code is executed.

next(c)

Then you use the next() function, the next function go to the next yield statement who is yield 1 here.
So this yield returns 1 and stops the generator's execution to try to catch a value.

The next line is
c.send(100), so the yield 1 was waiting for a value and you provided it, but this value isn't saved.

The send method also execute the rest of the generator until the next yield statement who is:

x = (yield 42)

So the yield here return 42 and stops the generator program to try to catch a value. But after that you call

next(c)

And you didn't provided a value so x is None now. Then the rest of the code is executed (until the next yield statement, don't forget)

print(x)
yield 2

So it prints x who is None and then the yield returns 2 and try to catch a value.

Try to write this

c = gen()
next(c)
next(c)
c.send(100)

And it will work (understand why !)

Python - Understanding the send function of a generator

From the documentation:

When send() is called to start the generator, it must be called with None as the argument, because there is no yield expression that could receive the value.

As for the exception, you can't really avoid it. The generator throws this exception when it's done iterating, so instead of avoiding it, just catch it:

g = f()

try:
    g.send(None)
    g.send('x')
    g.send('y')
except StopIteration:
    print 'Done'

How does Python work with multiple send calls to a generator?

Your generator function has three yield expressions, but you're throwing away the value from two of them (lines 5 and 6). If you did something with the values there, you'd see the 100 being used in the function. If you kept running your example, the fifth time you called send would cause the generator to update m to a new value.

Lets walk through the code that does the send calls in your example, and see what the generator is doing at the same time:

it = multiplier()

At this point the generator object has been created and saved to it. The generator code has not started running yet, it's paused at the start of the function's code.

print(str(it.send(None)))

This starts running the generator function's code. The value sent must be None or you'll get an error. The function never sees that value. It's more common to use next to start up a generator, since next(it) is equivalent to it.send(None).

The generator function runs until line 3, where the first yield appears. Since you're not yielding any particular value, the return value from send is None (which gets printed).

print(it.send(10))

This value gets sent to the generator and becomes the value of the yield expression on line 3. So 10 gets stored as m, and the code prints it out on line 4. The generator function keeps running to line 5, where it reaches the next yield expression. Since it's yielding str(m * 2), the calling code gets "20" and prints that.

print(it.send(100))

The 100 value gets sent into the generator as the value of the yield on line 4. That value is ignored, since you're not using the yield as an expression but as a statement. Just like putting 100 on a line by itself, this is perfectly legal, but maybe not very useful. The code goes on to line 5 where it yields str(m * 3), or "30", which gets printed by the calling code.

That's where your driving code stops, but the generator is still alive, and you could send more values to it (and get more values back). The next value you send to the generator would also be ignored, just like the 100 was, but the value after that would end up as a new m value when the while loop in the generator returned to the top and the line 3 yield was reached.

I suspect that some of your confusion with send in this code has to do with the fact that you're using yield both as an expression and as a statement. Probably you don't want to be doing both. Usually you'll either care about all the values being sent into the generator, or you don't care about any of them. If you want to yield several values together (like n*2 and n*3), you could yield a tuple rather than a single item.

Here's a modified version of your code that I think might be easier for you to play with and understand:

def multiplier():
    print("top of generator")
    m = yield # nothing to yield the first time, just a value we get
    print("before loop, m =", m)
    while True:
        print("top of loop, m =", m)
        m = yield m * 2, m * 3         # we always care about the value we're sent
        print("bottom of loop, m =", m)

print("calling generator")
it = multiplier()

print("calling next")
next(it)   # this is equivalent to it.send(None)

print("sending 10")
print(it.send(10))

print("sending 20")
print(it.send(20))

print("sending 100")
print(it.send(100))

Python's SendType and ReturnType in Generators

Generators have a send method:

generator.send(value)
Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value. When send() is called to start the generator, it must be called with None as the argument, because there is no yield expression that could receive the value.

See also What's the purpose of "send" function on Python generators?

So with your generator you can do this:

it = echo_round()
print(next(it))      # 0 -- we could also have printed: it.send(None) 
print(it.send(1.1))  # 1
print(it.send(2.5))  # 2
print(it.send(-1.8)) # StopIteration: Done

...but yield 0 returns None

...unless we call send after the first yield has been made, like in the above script. Here yield 0 will return 1.1

We see all three types at play here:

it.send(1.1) takes a value of the SendType (float in this case), and returns a value of the YieldType (int in this case), or raises a StopIteration error with a value of the ReturnType (str in this case).

What can you use generator functions for?

Generators give you lazy evaluation. You use them by iterating over them, either explicitly with 'for' or implicitly by passing it to any function or construct that iterates. You can think of generators as returning multiple items, as if they return a list, but instead of returning them all at once they return them one-by-one, and the generator function is paused until the next item is requested.

Generators are good for calculating large sets of results (in particular calculations involving loops themselves) where you don't know if you are going to need all results, or where you don't want to allocate the memory for all results at the same time. Or for situations where the generator uses another generator, or consumes some other resource, and it's more convenient if that happened as late as possible.

Another use for generators (that is really the same) is to replace callbacks with iteration. In some situations you want a function to do a lot of work and occasionally report back to the caller. Traditionally you'd use a callback function for this. You pass this callback to the work-function and it would periodically call this callback. The generator approach is that the work-function (now a generator) knows nothing about the callback, and merely yields whenever it wants to report something. The caller, instead of writing a separate callback and passing that to the work-function, does all the reporting work in a little 'for' loop around the generator.

For example, say you wrote a 'filesystem search' program. You could perform the search in its entirety, collect the results and then display them one at a time. All of the results would have to be collected before you showed the first, and all of the results would be in memory at the same time. Or you could display the results while you find them, which would be more memory efficient and much friendlier towards the user. The latter could be done by passing the result-printing function to the filesystem-search function, or it could be done by just making the search function a generator and iterating over the result.

If you want to see an example of the latter two approaches, see os.path.walk() (the old filesystem-walking function with callback) and os.walk() (the new filesystem-walking generator.) Of course, if you really wanted to collect all results in a list, the generator approach is trivial to convert to the big-list approach:

big_list = list(the_generator)

Where is the source code of python generator send?

Look here for class Generator, also look for this file, is a complete implementation of generators on C inside Python