What Does the "Yield" Keyword Do

What does the yield keyword do?

To understand what yield does, you must understand what generators are. And before you can understand generators, you must understand iterables.

Iterables

When you create a list, you can read its items one by one. Reading its items one by one is called iteration:

>>> mylist = [1, 2, 3]
>>> for i in mylist:
...    print(i)
1
2
3

mylist is an iterable. When you use a list comprehension, you create a list, and so an iterable:

>>> mylist = [x*x for x in range(3)]
>>> for i in mylist:
...    print(i)
0
1
4

Everything you can use "for... in..." on is an iterable; lists, strings, files...

These iterables are handy because you can read them as much as you wish, but you store all the values in memory and this is not always what you want when you have a lot of values.

Generators

Generators are iterators, a kind of iterable you can only iterate over once. Generators do not store all the values in memory, they generate the values on the fly:

>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
...    print(i)
0
1
4

It is just the same except you used () instead of []. BUT, you cannot perform for i in mygenerator a second time since generators can only be used once: they calculate 0, then forget about it and calculate 1, and end calculating 4, one by one.

Yield

yield is a keyword that is used like return, except the function will return a generator.

>>> def create_generator():
...    mylist = range(3)
...    for i in mylist:
...        yield i*i
...
>>> mygenerator = create_generator() # create a generator
>>> print(mygenerator) # mygenerator is an object!
<generator object create_generator at 0xb7555c34>
>>> for i in mygenerator:
...     print(i)
0
1
4

Here it's a useless example, but it's handy when you know your function will return a huge set of values that you will only need to read once.

To master yield, you must understand that when you call the function, the code you have written in the function body does not run. The function only returns the generator object, this is a bit tricky.

Then, your code will continue from where it left off each time for uses the generator.

Now the hard part:

The first time the for calls the generator object created from your function, it will run the code in your function from the beginning until it hits yield, then it'll return the first value of the loop. Then, each subsequent call will run another iteration of the loop you have written in the function and return the next value. This will continue until the generator is considered empty, which happens when the function runs without hitting yield. That can be because the loop has come to an end, or because you no longer satisfy an "if/else".

Your code explained

Generator:

# Here you create the method of the node object that will return the generator
def _get_child_candidates(self, distance, min_dist, max_dist):

    # Here is the code that will be called each time you use the generator object:

    # If there is still a child of the node object on its left
    # AND if the distance is ok, return the next child
    if self._leftchild and distance - max_dist < self._median:
        yield self._leftchild

    # If there is still a child of the node object on its right
    # AND if the distance is ok, return the next child
    if self._rightchild and distance + max_dist >= self._median:
        yield self._rightchild

    # If the function arrives here, the generator will be considered empty
    # there is no more than two values: the left and the right children

Caller:

# Create an empty list and a list with the current object reference
result, candidates = list(), [self]

# Loop on candidates (they contain only one element at the beginning)
while candidates:

    # Get the last candidate and remove it from the list
    node = candidates.pop()

    # Get the distance between obj and the candidate
    distance = node._get_dist(obj)

    # If distance is ok, then you can fill the result
    if distance <= max_dist and distance >= min_dist:
        result.extend(node._values)

    # Add the children of the candidate in the candidate's list
    # so the loop will keep running until it will have looked
    # at all the children of the children of the children, etc. of the candidate
    candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))

return result

This code contains several smart parts:

The loop iterates on a list, but the list expands while the loop is being iterated. It's a concise way to go through all these nested data even if it's a bit dangerous since you can end up with an infinite loop. In this case, candidates.extend(node._get_child_candidates(distance, min_dist, max_dist)) exhaust all the values of the generator, but while keeps creating new generator objects which will produce different values from the previous ones since it's not applied on the same node.
The extend() method is a list object method that expects an iterable and adds its values to the list.

Usually we pass a list to it:

>>> a = [1, 2]
>>> b = [3, 4]
>>> a.extend(b)
>>> print(a)
[1, 2, 3, 4]

But in your code, it gets a generator, which is good because:

You don't need to read the values twice.
You may have a lot of children and you don't want them all stored in memory.

And it works because Python does not care if the argument of a method is a list or not. Python expects iterables so it will work with strings, lists, tuples, and generators! This is called duck typing and is one of the reasons why Python is so cool. But this is another story, for another question...

You can stop here, or read a little bit to see an advanced use of a generator:

Controlling a generator exhaustion

>>> class Bank(): # Let's create a bank, building ATMs
...    crisis = False
...    def create_atm(self):
...        while not self.crisis:
...            yield "$100"
>>> hsbc = Bank() # When everything's ok the ATM gives you as much as you want
>>> corner_street_atm = hsbc.create_atm()
>>> print(corner_street_atm.next())
$100
>>> print(corner_street_atm.next())
$100
>>> print([corner_street_atm.next() for cash in range(5)])
['$100', '$100', '$100', '$100', '$100']
>>> hsbc.crisis = True # Crisis is coming, no more money!
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> wall_street_atm = hsbc.create_atm() # It's even true for new ATMs
>>> print(wall_street_atm.next())
<type 'exceptions.StopIteration'>
>>> hsbc.crisis = False # The trouble is, even post-crisis the ATM remains empty
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> brand_new_atm = hsbc.create_atm() # Build a new one to get back in business
>>> for cash in brand_new_atm:
...    print cash
$100
$100
$100
$100
$100
$100
$100
$100
$100
...

Note: For Python 3, useprint(corner_street_atm.__next__()) or print(next(corner_street_atm))

It can be useful for various things like controlling access to a resource.

Itertools, your best friend

The itertools module contains special functions to manipulate iterables. Ever wish to duplicate a generator?
Chain two generators? Group values in a nested list with a one-liner? Map / Zip without creating another list?

Then just import itertools.

An example? Let's see the possible orders of arrival for a four-horse race:

>>> horses = [1, 2, 3, 4]
>>> races = itertools.permutations(horses)
>>> print(races)
<itertools.permutations object at 0xb754f1dc>
>>> print(list(itertools.permutations(horses)))
[(1, 2, 3, 4),
 (1, 2, 4, 3),
 (1, 3, 2, 4),
 (1, 3, 4, 2),
 (1, 4, 2, 3),
 (1, 4, 3, 2),
 (2, 1, 3, 4),
 (2, 1, 4, 3),
 (2, 3, 1, 4),
 (2, 3, 4, 1),
 (2, 4, 1, 3),
 (2, 4, 3, 1),
 (3, 1, 2, 4),
 (3, 1, 4, 2),
 (3, 2, 1, 4),
 (3, 2, 4, 1),
 (3, 4, 1, 2),
 (3, 4, 2, 1),
 (4, 1, 2, 3),
 (4, 1, 3, 2),
 (4, 2, 1, 3),
 (4, 2, 3, 1),
 (4, 3, 1, 2),
 (4, 3, 2, 1)]

Understanding the inner mechanisms of iteration

Iteration is a process implying iterables (implementing the __iter__() method) and iterators (implementing the __next__() method).
Iterables are any objects you can get an iterator from. Iterators are objects that let you iterate on iterables.

There is more about it in this article about how for loops work.

What does the yield keyword do?

To understand what yield does, you must understand what generators are. And before you can understand generators, you must understand iterables.

Iterables

When you create a list, you can read its items one by one. Reading its items one by one is called iteration:

>>> mylist = [1, 2, 3]
>>> for i in mylist:
...    print(i)
1
2
3

mylist is an iterable. When you use a list comprehension, you create a list, and so an iterable:

>>> mylist = [x*x for x in range(3)]
>>> for i in mylist:
...    print(i)
0
1
4

Everything you can use "for... in..." on is an iterable; lists, strings, files...

These iterables are handy because you can read them as much as you wish, but you store all the values in memory and this is not always what you want when you have a lot of values.

Generators

Generators are iterators, a kind of iterable you can only iterate over once. Generators do not store all the values in memory, they generate the values on the fly:

>>> mygenerator = (x*x for x in range(3))
>>> for i in mygenerator:
...    print(i)
0
1
4

Yield

yield is a keyword that is used like return, except the function will return a generator.

>>> def create_generator():
...    mylist = range(3)
...    for i in mylist:
...        yield i*i
...
>>> mygenerator = create_generator() # create a generator
>>> print(mygenerator) # mygenerator is an object!
<generator object create_generator at 0xb7555c34>
>>> for i in mygenerator:
...     print(i)
0
1
4

Here it's a useless example, but it's handy when you know your function will return a huge set of values that you will only need to read once.

Then, your code will continue from where it left off each time for uses the generator.

Now the hard part:

Your code explained

Generator:

# Here you create the method of the node object that will return the generator
def _get_child_candidates(self, distance, min_dist, max_dist):

    # Here is the code that will be called each time you use the generator object:

    # If there is still a child of the node object on its left
    # AND if the distance is ok, return the next child
    if self._leftchild and distance - max_dist < self._median:
        yield self._leftchild

    # If there is still a child of the node object on its right
    # AND if the distance is ok, return the next child
    if self._rightchild and distance + max_dist >= self._median:
        yield self._rightchild

    # If the function arrives here, the generator will be considered empty
    # there is no more than two values: the left and the right children

Caller:

# Create an empty list and a list with the current object reference
result, candidates = list(), [self]

# Loop on candidates (they contain only one element at the beginning)
while candidates:

    # Get the last candidate and remove it from the list
    node = candidates.pop()

    # Get the distance between obj and the candidate
    distance = node._get_dist(obj)

    # If distance is ok, then you can fill the result
    if distance <= max_dist and distance >= min_dist:
        result.extend(node._values)

    # Add the children of the candidate in the candidate's list
    # so the loop will keep running until it will have looked
    # at all the children of the children of the children, etc. of the candidate
    candidates.extend(node._get_child_candidates(distance, min_dist, max_dist))

return result

This code contains several smart parts:

The loop iterates on a list, but the list expands while the loop is being iterated. It's a concise way to go through all these nested data even if it's a bit dangerous since you can end up with an infinite loop. In this case, candidates.extend(node._get_child_candidates(distance, min_dist, max_dist)) exhaust all the values of the generator, but while keeps creating new generator objects which will produce different values from the previous ones since it's not applied on the same node.
The extend() method is a list object method that expects an iterable and adds its values to the list.

Usually we pass a list to it:

>>> a = [1, 2]
>>> b = [3, 4]
>>> a.extend(b)
>>> print(a)
[1, 2, 3, 4]

But in your code, it gets a generator, which is good because:

You don't need to read the values twice.
You may have a lot of children and you don't want them all stored in memory.

You can stop here, or read a little bit to see an advanced use of a generator:

Controlling a generator exhaustion

>>> class Bank(): # Let's create a bank, building ATMs
...    crisis = False
...    def create_atm(self):
...        while not self.crisis:
...            yield "$100"
>>> hsbc = Bank() # When everything's ok the ATM gives you as much as you want
>>> corner_street_atm = hsbc.create_atm()
>>> print(corner_street_atm.next())
$100
>>> print(corner_street_atm.next())
$100
>>> print([corner_street_atm.next() for cash in range(5)])
['$100', '$100', '$100', '$100', '$100']
>>> hsbc.crisis = True # Crisis is coming, no more money!
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> wall_street_atm = hsbc.create_atm() # It's even true for new ATMs
>>> print(wall_street_atm.next())
<type 'exceptions.StopIteration'>
>>> hsbc.crisis = False # The trouble is, even post-crisis the ATM remains empty
>>> print(corner_street_atm.next())
<type 'exceptions.StopIteration'>
>>> brand_new_atm = hsbc.create_atm() # Build a new one to get back in business
>>> for cash in brand_new_atm:
...    print cash
$100
$100
$100
$100
$100
$100
$100
$100
$100
...

Note: For Python 3, useprint(corner_street_atm.__next__()) or print(next(corner_street_atm))

It can be useful for various things like controlling access to a resource.

Itertools, your best friend

Then just import itertools.

An example? Let's see the possible orders of arrival for a four-horse race:

>>> horses = [1, 2, 3, 4]
>>> races = itertools.permutations(horses)
>>> print(races)
<itertools.permutations object at 0xb754f1dc>
>>> print(list(itertools.permutations(horses)))
[(1, 2, 3, 4),
 (1, 2, 4, 3),
 (1, 3, 2, 4),
 (1, 3, 4, 2),
 (1, 4, 2, 3),
 (1, 4, 3, 2),
 (2, 1, 3, 4),
 (2, 1, 4, 3),
 (2, 3, 1, 4),
 (2, 3, 4, 1),
 (2, 4, 1, 3),
 (2, 4, 3, 1),
 (3, 1, 2, 4),
 (3, 1, 4, 2),
 (3, 2, 1, 4),
 (3, 2, 4, 1),
 (3, 4, 1, 2),
 (3, 4, 2, 1),
 (4, 1, 2, 3),
 (4, 1, 3, 2),
 (4, 2, 1, 3),
 (4, 2, 3, 1),
 (4, 3, 1, 2),
 (4, 3, 2, 1)]

Understanding the inner mechanisms of iteration

There is more about it in this article about how for loops work.

What does 'yield' keyword do in flutter?

yield adds a value to the output stream of the surrounding async* function. It's like return, but doesn't terminate the function.

See https://dart.dev/guides/language/language-tour#generators

Stream asynchronousNaturalsTo(n) async* {
  int k = 0;
  while (k < n) yield k++;
}

When the yield statement executes, it adds the result of evaluating its expression to the stream. It doesn’t necessarily suspend (though in the current implementations it does).

What's the yield keyword in JavaScript?

The MDN documentation is pretty good, IMO.

The function containing the yield keyword is a generator. When you call it, its formal parameters are bound to actual arguments, but its body isn't actually evaluated. Instead, a generator-iterator is returned. Each call to the generator-iterator's next() method performs another pass through the iterative algorithm. Each step's value is the value specified by the yield keyword. Think of yield as the generator-iterator version of return, indicating the boundary between each iteration of the algorithm. Each time you call next(), the generator code resumes from the statement following the yield.

What is the yield keyword used for in C#?

The yield contextual keyword actually does quite a lot here.

The function returns an object that implements the IEnumerable<object> interface. If a calling function starts foreaching over this object, the function is called again until it "yields". This is syntactic sugar introduced in C# 2.0. In earlier versions you had to create your own IEnumerable and IEnumerator objects to do stuff like this.

The easiest way understand code like this is to type-in an example, set some breakpoints and see what happens. Try stepping through this example:

public void Consumer()
{
    foreach(int i in Integers())
    {
        Console.WriteLine(i.ToString());
    }
}

public IEnumerable<int> Integers()
{
    yield return 1;
    yield return 2;
    yield return 4;
    yield return 8;
    yield return 16;
    yield return 16777216;
}

When you step through the example, you'll find the first call to Integers() returns 1. The second call returns 2 and the line yield return 1 is not executed again.

Here is a real-life example:

public IEnumerable<T> Read<T>(string sql, Func<IDataReader, T> make, params object[] parms)
{
    using (var connection = CreateConnection())
    {
        using (var command = CreateCommand(CommandType.Text, sql, connection, parms))
        {
            command.CommandTimeout = dataBaseSettings.ReadCommandTimeout;
            using (var reader = command.ExecuteReader())
            {
                while (reader.Read())
                {
                    yield return make(reader);
                }
            }
        }
    }
}

What does yield mean in PHP?

What is `yield`?

The yield keyword returns data from a generator function:

The heart of a generator function is the yield keyword. In its simplest form, a yield statement looks much like a return statement, except that instead of stopping execution of the function and returning, yield instead provides a value to the code looping over the generator and pauses execution of the generator function.

What is a generator function?

A generator function is effectively a more compact and efficient way to write an Iterator. It allows you to define a function (your xrange) that will calculate and return values while you are looping over it:

function xrange($min, $max) {
    for ($i = $min; $i <= $max; $i++) {
        yield $i;
    }
}

[…]

foreach (xrange(1, 10) as $key => $value) {
    echo "$key => $value", PHP_EOL;
}

This would create the following output:

0 => 1
1 => 2
…
9 => 10

You can also control the $key in the foreach by using

yield $someKey => $someValue;

In the generator function, $someKey is whatever you want appear for $key and $someValue being the value in $val. In the question's example that's $i.

What's the difference to normal functions?

Now you might wonder why we are not simply using PHP's native range function to achieve that output. And right you are. The output would be the same. The difference is how we got there.

When we use range PHP, will execute it, create the entire array of numbers in memory and return that entire array to the foreach loop which will then go over it and output the values. In other words, the foreach will operate on the array itself. The range function and the foreach only "talk" once. Think of it like getting a package in the mail. The delivery guy will hand you the package and leave. And then you unwrap the entire package, taking out whatever is in there.

When we use the generator function, PHP will step into the function and execute it until it either meets the end or a yield keyword. When it meets a yield, it will then return whatever is the value at that time to the outer loop. Then it goes back into the generator function and continues from where it yielded. Since your xrange holds a for loop, it will execute and yield until $max was reached. Think of it like the foreach and the generator playing ping pong.

Why do I need that?

Obviously, generators can be used to work around memory limits. Depending on your environment, doing a range(1, 1000000) will fatal your script whereas the same with a generator will just work fine. Or as Wikipedia puts it:

Because generators compute their yielded values only on demand, they are useful for representing sequences that would be expensive or impossible to compute at once. These include e.g. infinite sequences and live data streams.

Generators are also supposed to be pretty fast. But keep in mind that when we are talking about fast, we are usually talking in very small numbers. So before you now run off and change all your code to use generators, do a benchmark to see where it makes sense.

Another Use Case for Generators is asynchronous coroutines. The yield keyword does not only return values but it also accepts them. For details on this, see the two excellent blog posts linked below.

Since when can I use `yield`?

Generators have been introduced in PHP 5.5. Trying to use yield before that version will result in various parse errors, depending on the code that follows the keyword. So if you get a parse error from that code, update your PHP.

Sources and further reading:

Official docs
The original RFC
kelunik's blog: An introduction to generators
ircmaxell's blog: What generators can do for you
NikiC's blog: Cooperative multitasking using coroutines in PHP
Co-operative PHP Multitasking
What is the difference between a generator and an array?
Wikipedia on Generators in general

What does a plain yield keyword do in Python?

Although they're almost always used as simple generators, yield enables fully-fledged coroutines.

Besides being used to send values out of a generator / co-routine, yield can also receive values, with the effect of coordinating the execution different co-routines. Thus, you don't need expression_list if you only want to coordinate or receive values.

Here's a simple example:

def infini_printer():
    while True:
        received = yield # get the received value into a variable
        print(received)

printer = infini_printer()
printer.next()              # returns None, since we didn't specify any expression after yield
printer.send("foo")         # prints "foo"

What does the yield keyword do in Ruby?

This is an example fleshing out your sample code:

class MyClass
  attr_accessor :items

  def initialize(ary=[])
    @items = ary
  end

  def each
    @items.each do |item| 
      yield item
    end
  end
end

my_class = MyClass.new(%w[a b c d])
my_class.each do |y|
  puts y
end
# >> a
# >> b
# >> c
# >> d

each loops over a collection. In this case it's looping over each item in the @items array, initialized/created when I did the new(%w[a b c d]) statement.

yield item in the MyClass.each method passes item to the block attached to my_class.each. The item being yielded is assigned to the local y.

Does that help?

Now, here's a bit more about how each works. Using the same class definition, here's some code:

my_class = MyClass.new(%w[a b c d])

# This points to the `each` Enumerator/method of the @items array in your instance via
#  the accessor you defined, not the method "each" you've defined.
my_class_iterator = my_class.items.each # => #<Enumerator: ["a", "b", "c", "d"]:each>

# get the next item on the array
my_class_iterator.next # => "a"

# get the next item on the array
my_class_iterator.next # => "b"

# get the next item on the array
my_class_iterator.next # => "c"

# get the next item on the array
my_class_iterator.next # => "d"

# get the next item on the array
my_class_iterator.next # => 
# ~> -:21:in `next': iteration reached an end (StopIteration)
# ~>    from -:21:in `<main>'

Notice that on the last next the iterator fell off the end of the array. This is the potential pitfall for NOT using a block because if you don't know how many elements are in the array you can ask for too many items and get an exception.

Using each with a block will iterate over the @items receiver and stop when it reaches the last item, avoiding the error, and keeping things nice and clean.

How does this python yield function work?

A yield statement is used like return to return a value but it doesn't destroy the stack frame (the part of a function that knows the current line, local variables, and pending try-statements). This allows the function to be resumed after the yield.

When you call a function containing yield, it returns a "generator" that allows you to run code up to a yield and then to resume it from where it left off.

>>> def squares(n):
        for i in range(n):
            yield i ** 2

>>> g = squares(5)             # create the generator
>>> g
<generator object squares at 0x106beef10>
>>> next(g)                    # run until the first yield
0
>>> next(g)                    # resume after the yield
1
>>> next(g)                    # resume after the yield
4
>>> next(g)                    # resume after the yield
9
>>> next(g)                    # resume after the yield
16
>>> next(g)                    # looping is terminated with a StopIteration
Traceback (most recent call last):
  File "<pyshell#13>", line 1, in <module>
    next(g)                    # looping is terminated with a StopIteration
StopIteration

Interestingly, a generator can accept values using the sen

Related Topics

Staleelementreferenceexception on Python Selenium
What Do I Need to Read Microsoft Access Databases Using Python
Use the Default Python Rather Than the Anaconda Installation When Called from the Terminal
Ruby Equivalent of Virtualenv
"Is" Operator Behaves Unexpectedly With Integers
Accessing Class Variables from a List Comprehension in the Class Definition
What Does the Star and Doublestar Operator Mean in a Function Call
What's the Canonical Way to Check For Type in Python
Check If Multiple Strings Exist in Another String
What Does "Syntaxerror: Missing Parentheses in Call to 'Print'" Mean in Python
Selenium.Common.Exceptions.Invalidselectorexception With "Span:Contains('String')"
Process List on Linux Via Python
How to Install Pyodbc on Linux
Regexp Finding Longest Common Prefix of Two Strings
How to Merge Two Dictionaries in a Single Expression
Why Isn't the 'Global' Keyword Needed to Access a Global Variable
Transpose List of Lists
Difference Between Shallow Copy, Deepcopy and Normal Assignment Operation

What does the yield keyword do?

Iterables

Generators

Yield

Your code explained

Controlling a generator exhaustion

Itertools, your best friend

Understanding the inner mechanisms of iteration

What does the yield keyword do?

Iterables

Generators

Yield

Your code explained

Controlling a generator exhaustion

Itertools, your best friend

Understanding the inner mechanisms of iteration

What does 'yield' keyword do in flutter?

What's the yield keyword in JavaScript?

What is the yield keyword used for in C#?

What does yield mean in PHP?

What is yield?

What is a generator function?

What's the difference to normal functions?

Why do I need that?

Since when can I use yield?

Sources and further reading:

What does a plain yield keyword do in Python?

What does the yield keyword do in Ruby?

How does this python yield function work?

Related Topics

Leave a reply

What is `yield`?

Since when can I use `yield`?