Python Iterator Is Empty After Performing Some Action on It

Python iterator is empty after performing some action on it

This is exactly how iterators work. They're designed to generate data on the fly exactly one time, no more. If you want to get data out of it a second time, you either have to save all the iterated data to another list, or initiate the iterator again. In cases where you need to read from files or do other annoying things to re-obtain that iterator, it's probably best to just store that data in a list when it's generated.

>>> numbers = ('1','2','3','4','5')
>>> ints = [x for x in map(int, numbers)]
>>> print(list(ints))
[1, 2, 3, 4, 5]
>>> print(list(ints))
[1, 2, 3, 4, 5]

https://docs.python.org/2/library/stdtypes.html#iterator-types

The intention of the protocol is that once an iterator’s next() method
raises StopIteration, it will continue to do so on subsequent calls.
Implementations that do not obey this property are deemed broken.
(This constraint was added in Python 2.3; in Python 2.2, various
iterators are broken according to this rule.)

I should note that I ran the exact code you gave on Python 2.4.3, and it printed out the list every time. So it's a version dependent thing, I suppose.

Loop over empty iterator does not raise exception

There is no builtin way to directly check for empty iterators. Empty iterators are generally not considered exceptional. However, you can define compact helpers to explicitly check for empty iterators.


Detecting empty iterators up-front is generally not possible. In some cases, one can guess from the source of an iterator whether it is empty -- for example, iterators of sequences and mappings are empty if their parent is boolean false. However, iterators themselves do not have any indication for content and may in fact be "empty" unless iterated over.

The only reliable means is to check whether an iterator provides items while iterating.
One can define a wrapper iterator that raises an error if an iterator provides no items at all.

def non_empty(iterable):
"""Helper to ensure that ``iterable`` is not empty during iteration"""
iterator = iter(iterable)
try:
yield next(iterator) # explicitly check first item
except StopIteration:
raise LookupError(f'{iterable} is empty') from None
yield from iterator # forward iteration of later items

Such a helper can be wrapped around both iterables and iterators, and works in for loops, explicit iter iterators and any other iteration scenario.

>>> iterable = []
>>> for element in non_empty(iterable):
... assert isinstance(element, int)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in non_empty
LookupError: [] is empty

Why are some (but not all) Python iterators summable after being exhausted?

The problem is that the custom iterator is initialising inside the __iter__ method. Even though i2 = iter(CustomIterator()) includes an explicit call to iter, the sum function (and also min, max, for, etc) will still call i2.__iter__() again and reset i2.

There's a bunch of tutorials out there on "how to make Python iterators", and about half of them say something like "to make an iterator, you just have to define iter and next methods". While this is technically correct as per the documentation, it will get you into trouble sometimes. In many cases you'll also want a separate __init__ method to initialise the iterator.

So to fix this problem, redefine CustomIterator as:

class CustomIterator:
def __init__(self):
self.n=0

def __iter__(self):
return self

def __next__(self):
self.n += 1
if self.n > 3:
raise StopIteration
return self.n

i1 = iter([1,2,3])
i2 = CustomIterator() ### iter(...) is not needed here (but won't do any harm either)

Then init is called once and once only on creating a new iterator, and repeated calls to iter won't reset the iterator.

Testing for an empty iterator in a Python for... loop

You don't need to check if the iterator is empty. The for loop will do that for you, and stop when the iterator is empty. It's as simple as that.

Also, you don't need the else after the sys.exit() or the return.

That done, your code looks like this:

def my_func(choice_pattern, input):
# Search in input for some things to choose from.
choice_iterator = choice_pattern.finditer(input, re.M)
if not choice_iterator:
print "No choices. Exiting..."
sys.exit()

# Show choices to the user. For each one, ask user for a yes/no response. If
# choice accepted, return a result. Otherwise show user next choice. If no
# choices accepted by user, quit.
for choice in choice_iterator:
print choice
# get_yes_no_answer() returns True or False depending on user response.
if get_yes_no_answer():
return choice
# Loop exited without matches.
print "No matches. Exiting..."
sys.exit()

That's it!

What happens is that you in the loop, also gets the next item. The result is that you in fact only show every second answer.

In fact, you can simplify it even more:

def my_func(choice_pattern, input):
choice_iterator = choice_pattern.finditer(input, re.M)
if choice_iterator:
for choice in choice_iterator:
print choice
if get_yes_no_answer():
return choice
# If there is no choices or no matches, you end up here:
print "No matches. Exiting..."
sys.exit()

Iterators are used pretty much as any sequence type. You don't need to treat it differently from a list.

Why I'm getting empty list after using zip in python?

zip returns an iterator object. The first time you convert it to a list, the iterator is consumed, and after that point it's empty. You'll get the behavior you expect if you convert it to a list immediately and then copy it:

result = list(zip(number_list, str_list))

# Converting list to more lists
result_list = result.copy()
result_list2 = result.copy()

In general, a zip object is meant to be used immediately (e.g. within a for loop or passed directly to a function that takes an iterable). Assigning the result of calling zip to a variable is not going to be useful in very many cases.

Python: list(filter object) empties the object

A filter is a special iterable object, and like a generator, you can only iterate over it once. So essentially it returns an empty list when you run it a second time.

re.finditer has some length but I'm unable to iterate over it

re.finditer, as the name implies, returns an iterator. When you use len(list(out)) you consume the iterator, and from then on it is empty.

You can test it like so:

assert len(list(out)) == 2 # First will work
assert len(list(out)) == 2 # Second won't

Just remove the assert line and you'll receive output:

out = re.finditer(r'TP', s)

# We love output :D
for m in out:
print(m)

For more info, you can consult this question.



Related Topics



Leave a reply



Submit