How to Write a Generator Class

How to write a generator class?

How to write a generator class?

You're almost there, writing an Iterator class (I show a Generator at the end of the answer), but __next__ gets called every time you call the object with next, returning a generator object. Instead, to make your code work with the least changes, and the fewest lines of code, use __iter__, which makes your class instantiate an iterable (which isn't technically a generator):

class Fib:
    def __init__(self):
        self.a, self.b = 0, 1
    def __iter__(self):
        while True:
            yield self.a
            self.a, self.b = self.b, self.a+self.b

When we pass an iterable to iter(), it gives us an iterator:

>>> f = iter(Fib())
>>> for i in range(3):
...     print(next(f))
...
0
1
1

To make the class itself an iterator, it does require a __next__:

class Fib:
    def __init__(self):
        self.a, self.b = 0, 1        
    def __next__(self):
        return_value = self.a
        self.a, self.b = self.b, self.a+self.b
        return return_value
    def __iter__(self):
        return self

And now, since iter just returns the instance itself, we don't need to call it:

>>> f = Fib()
>>> for i in range(3):
...     print(next(f))
...
0
1
1

Why is the value self.a not getting printed?

Here's your original code with my comments:

class Fib:
    def __init__(self):
        self.a, self.b = 0, 1
        
    def __next__(self):
        yield self.a          # yield makes .__next__() return a generator!
        self.a, self.b = self.b, self.a+self.b

f = Fib()

for i in range(3):
    print(next(f))

So every time you called next(f) you got the generator object that __next__ returns:

<generator object __next__ at 0x000000000A3E4F68>
<generator object __next__ at 0x000000000A3E4F68>
<generator object __next__ at 0x000000000A3E4F68>

Also, how do I write unittest for generators?

You still need to implement a send and throw method for a Generator

from collections.abc import Iterator, Generator
import unittest

class Test(unittest.TestCase):
    def test_Fib(self):
        f = Fib()
        self.assertEqual(next(f), 0)
        self.assertEqual(next(f), 1)
        self.assertEqual(next(f), 1)
        self.assertEqual(next(f), 2) #etc...
    def test_Fib_is_iterator(self):
        f = Fib()
        self.assertIsInstance(f, Iterator)
    def test_Fib_is_generator(self):
        f = Fib()
        self.assertIsInstance(f, Generator)

And now:

>>> unittest.main(exit=False)
..F
======================================================================
FAIL: test_Fib_is_generator (__main__.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<stdin>", line 7, in test_Fib_is_generator
AssertionError: <__main__.Fib object at 0x00000000031A6320> is not an instance of <class 'collections.abc.Generator'>

----------------------------------------------------------------------
Ran 3 tests in 0.001s

FAILED (failures=1)
<unittest.main.TestProgram object at 0x0000000002CAC780>

So let's implement a generator object, and leverage the Generator abstract base class from the collections module (see the source for its implementation), which means we only need to implement send and throw - giving us close, __iter__ (returns self), and __next__ (same as .send(None)) for free (see the Python data model on coroutines):

class Fib(Generator):
    def __init__(self):
        self.a, self.b = 0, 1        
    def send(self, ignored_arg):
        return_value = self.a
        self.a, self.b = self.b, self.a+self.b
        return return_value
    def throw(self, type=None, value=None, traceback=None):
        raise StopIteration
    

and using the same tests above:

>>> unittest.main(exit=False)
...
----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK
<unittest.main.TestProgram object at 0x00000000031F7CC0>

Python 2

The ABC Generator is only in Python 3. To do this without Generator, we need to write at least close, __iter__, and __next__ in addition to the methods we defined above.

class Fib(object):
    def __init__(self):
        self.a, self.b = 0, 1        
    def send(self, ignored_arg):
        return_value = self.a
        self.a, self.b = self.b, self.a+self.b
        return return_value
    def throw(self, type=None, value=None, traceback=None):
        raise StopIteration
    def __iter__(self):
        return self
    def next(self):
        return self.send(None)
    def close(self):
        """Raise GeneratorExit inside generator.
        """
        try:
            self.throw(GeneratorExit)
        except (GeneratorExit, StopIteration):
            pass
        else:
            raise RuntimeError("generator ignored GeneratorExit")

Note that I copied close directly from the Python 3 standard library, without modification.

How can I write a generator in a JavaScript class?

Edit: Add more examples.

Your class definition is (almost) correct.The error was in instantiation var Reply = new Reply();. This tries to redefine variable assigned to class name. Also generator function is expected to yield something. I elaborated a little OP code to show working example.

class Reply {
  //added for test purpose
  constructor(...args) {
      this.args = args;
    }
    * getReply(msg) {
      for (let arg in this.args) {
        let reply = msg + this.args[arg];
        //generator should yield something
        yield reply;
      }
      //next call returns (yields) {done:true,value:undefined}
    }
    * otherFun() {
      yield this.getReply('Nice to meet you '); //yields Generator object
      yield this.getReply('See you '); //Yes, this can access 
      //next call yields {done:true, value:undefined}
    }
    * evenMore() {
      yield* this.getReply('I miss you '); //yields generator result(s)
      yield* this.getReply('I miss you even more ');
    }
}
//now test what we have
const reply = new Reply('Peter', 'James', 'John');
//let and var here are interchangeable because of Global scope
var r = reply.getReply('Hello ');
var msg = r.next(); //{done:false,value:"..."}
while (!msg.done) {
  console.log(msg.value);
  msg = r.next();
}
var other = reply.otherFun();
var o = other.next(); //{done:false,value:Generator}
while (!o.done) {
  let gen = o.value;
  msg = gen.next();
  while (!msg.done) {
    console.log(msg.value);
    msg = gen.next();
  }
  o = other.next();
}
var more = reply.evenMore();
msg = more.next();
while (!msg.done) {
  console.log(msg.value);
  msg = more.next();
}
//update of 1/12/2019
//more examples
for (let r of reply.getReply('for ')) {
  console.log(r);
}
for (let r of reply.evenMore()) {
  console.log(r);
}
//note that the following doesn't work because of lack of star (*) inside the generator function
for (let r of reply.otherFun()) {
  console.log(r);
}

How to create a custom generator class that is correctly garbage collected

PEP342, states:

[generator].__del__() is a wrapper for [generator].close(). This will be called when the generator object is garbage-collected ...

The Generator class in collections.abc does not implement __del__, and neither do its superclasses or metaclass.

Adding this implementation of __del__ to the class in the question results in the lock being freed:

class CustomGeneratorClass(Generator):

    ...

    def __del__(self):
        self.close() 

Output:

Generator Class Initialised: I grabbed a lock
Recieved  0
Recieved  1
Recieved  2
Recieved  3
Exception Thrown in Generator: I let go of the lock
Finished: Lock was free

Caveat:

I'm not experienced with the intricacies of object finalisation in Python, so this suggestion should be examined critically, and tested to destruction. In particular, the warnings about __del__ in the language reference should be considered.

---

A higher-level solution would be to run the generator in a context manager

with contextlib.closing(CustomGeneratorClass(100, lock)):
    # do stuff

but this is cumbersome, and relies on users of the code remembering to do it.

Function generators vs class generators in Python 3

Calling the f_gen() function produces an iterator (specifically, a generator iterator). Iterators can only ever be looped over once. Your class is not an iterator, it is instead an iterable, an object that can produce any number of iterators.

Your class produces a new generator iterator each time you use for, because for applies the iter() function on the object you pass in, which in turn calls object.__iter__(), which in your implementation returns a new generator iterator each time it is called.

In other words, you can make the class behave the same way by calling iter(instance) or instance.__iter__() before looping:

c_gen = CCounter(5,10)
c_gen_iterator = iter(c_gen)
for i in c_gen_iterator:
    # ...

You can also make the CCounter() into an iterator by returning self from __iter__, and adding an object.__next__() method (object.next() in Python 2):

class CCounter(object):
    def __init__(self, low, high):
        self.low = low
        self.high = high
    def __iter__(self):
       return self
    def __next__(self):
       result = self.low
       if result >= self.high:
           raise StopIteration()
       self.low += 1
       return result

Class generator definition

"Generator" is a specific built-in type, and your type will never be that type, no matter what methods you implement or what name you give it. You cannot make instances of your class be actual generators. (Not even through subclassing - the real generator type cannot be subclassed.) Also, generators have send and throw methods used for coroutine functionality, but it doesn't make sense for your class to have those, and implementing them still won't make your objects pass inspect.isgenerator.

inspect.isgenerator is intended to check for an actual generator, not arbitrary iterators or objects mimicking generators.

If the third-party library you're using needs a real generator, you will have to give it an actual generator, by writing and calling a generator function, or by writing a generator expression.

If the third-party library doesn't actually need anything generator-specific, you could file an issue asking that they switch to a less restrictive check than inspect.isgenerator.

Can a method within a class be generator?

Yes, this is perfectly normal. For example, it is commonly used to implement an object.__iter__() method to make an object an iterable:

class SomeContainer(object):
    def __iter__(self):
        for elem in self._datastructure:
            if elem.visible:
                yield elem.value

However, don't feel limited by that common pattern; anything that requires iteration is a candidate for a generator method.

When I create a class based generator, why do I have to call the next?

Your __next__ method itself is a generator function due to using yield. It must be a regular function that uses return instead.

def __next__(self):
    if self.count < self.max_times:
        self.count += 1
        return self.value   # return to provide one value on call
    raise StopIteration     # raise to end iteration

---

When iterating, python calls iter.__next__ to receive the new value. If this is a generator function, the call merely returns a generator. This is the same behaviour as for any other generator function:

>>> def foo():
...    yield 1
...
>>> foo()
<generator object foo at 0x106134ca8>

This requires you to call next on the generator to actually get a value. Similarly, since you defined BoundedGenerator.__next__ as a generator function, each iteration step provides only a new generator.

Using return instead of yield indeed returns the value, not a generator yielding said value. Additionally, you should raise StopIteration when done - this signals the end of the iteration.

---

Related Topics