Python iterator is empty after performing some action on it
This is exactly how iterators work. They're designed to generate data on the fly exactly one time, no more. If you want to get data out of it a second time, you either have to save all the iterated data to another list, or initiate the iterator again. In cases where you need to read from files or do other annoying things to re-obtain that iterator, it's probably best to just store that data in a list when it's generated.
>>> numbers = ('1','2','3','4','5')
>>> ints = [x for x in map(int, numbers)]
>>> print(list(ints))
[1, 2, 3, 4, 5]
>>> print(list(ints))
[1, 2, 3, 4, 5]
https://docs.python.org/2/library/stdtypes.html#iterator-types
The intention of the protocol is that once an iterator’s next() method
raises StopIteration, it will continue to do so on subsequent calls.
Implementations that do not obey this property are deemed broken.
(This constraint was added in Python 2.3; in Python 2.2, various
iterators are broken according to this rule.)
I should note that I ran the exact code you gave on Python 2.4.3, and it printed out the list every time. So it's a version dependent thing, I suppose.
Loop over empty iterator does not raise exception
There is no builtin way to directly check for empty iterators. Empty iterators are generally not considered exceptional. However, you can define compact helpers to explicitly check for empty iterators.
Detecting empty iterators up-front is generally not possible. In some cases, one can guess from the source of an iterator whether it is empty -- for example, iterators of sequences and mappings are empty if their parent is boolean false. However, iterators themselves do not have any indication for content and may in fact be "empty" unless iterated over.
The only reliable means is to check whether an iterator provides items while iterating.
One can define a wrapper iterator that raises an error if an iterator provides no items at all.
def non_empty(iterable):
"""Helper to ensure that ``iterable`` is not empty during iteration"""
iterator = iter(iterable)
try:
yield next(iterator) # explicitly check first item
except StopIteration:
raise LookupError(f'{iterable} is empty') from None
yield from iterator # forward iteration of later items
Such a helper can be wrapped around both iterables and iterators, and works in for
loops, explicit iter
iterators and any other iteration scenario.
>>> iterable = []
>>> for element in non_empty(iterable):
... assert isinstance(element, int)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 6, in non_empty
LookupError: [] is empty
Why are some (but not all) Python iterators summable after being exhausted?
The problem is that the custom iterator is initialising inside the __iter__
method. Even though i2 = iter(CustomIterator())
includes an explicit call to iter
, the sum
function (and also min
, max
, for
, etc) will still call i2.__iter__()
again and reset i2
.
There's a bunch of tutorials out there on "how to make Python iterators", and about half of them say something like "to make an iterator, you just have to define iter
and next
methods". While this is technically correct as per the documentation, it will get you into trouble sometimes. In many cases you'll also want a separate __init__
method to initialise the iterator.
So to fix this problem, redefine CustomIterator
as:
class CustomIterator:
def __init__(self):
self.n=0
def __iter__(self):
return self
def __next__(self):
self.n += 1
if self.n > 3:
raise StopIteration
return self.n
i1 = iter([1,2,3])
i2 = CustomIterator() ### iter(...) is not needed here (but won't do any harm either)
Then init
is called once and once only on creating a new iterator, and repeated calls to iter
won't reset the iterator.
Testing for an empty iterator in a Python for... loop
You don't need to check if the iterator is empty. The for loop will do that for you, and stop when the iterator is empty. It's as simple as that.
Also, you don't need the else
after the sys.exit() or the return.
That done, your code looks like this:
def my_func(choice_pattern, input):
# Search in input for some things to choose from.
choice_iterator = choice_pattern.finditer(input, re.M)
if not choice_iterator:
print "No choices. Exiting..."
sys.exit()
# Show choices to the user. For each one, ask user for a yes/no response. If
# choice accepted, return a result. Otherwise show user next choice. If no
# choices accepted by user, quit.
for choice in choice_iterator:
print choice
# get_yes_no_answer() returns True or False depending on user response.
if get_yes_no_answer():
return choice
# Loop exited without matches.
print "No matches. Exiting..."
sys.exit()
That's it!
What happens is that you in the loop, also gets the next item. The result is that you in fact only show every second answer.
In fact, you can simplify it even more:
def my_func(choice_pattern, input):
choice_iterator = choice_pattern.finditer(input, re.M)
if choice_iterator:
for choice in choice_iterator:
print choice
if get_yes_no_answer():
return choice
# If there is no choices or no matches, you end up here:
print "No matches. Exiting..."
sys.exit()
Iterators are used pretty much as any sequence type. You don't need to treat it differently from a list.
Why I'm getting empty list after using zip in python?
zip
returns an iterator object. The first time you convert it to a list, the iterator is consumed, and after that point it's empty. You'll get the behavior you expect if you convert it to a list immediately and then copy
it:
result = list(zip(number_list, str_list))
# Converting list to more lists
result_list = result.copy()
result_list2 = result.copy()
In general, a zip
object is meant to be used immediately (e.g. within a for
loop or passed directly to a function that takes an iterable). Assigning the result of calling zip
to a variable is not going to be useful in very many cases.
Python: list(filter object) empties the object
A filter is a special iterable object, and like a generator, you can only iterate over it once. So essentially it returns an empty list when you run it a second time.
re.finditer has some length but I'm unable to iterate over it
re.finditer
, as the name implies, returns an iterator. When you use len(list(out))
you consume the iterator, and from then on it is empty.
You can test it like so:
assert len(list(out)) == 2 # First will work
assert len(list(out)) == 2 # Second won't
Just remove the assert
line and you'll receive output:
out = re.finditer(r'TP', s)
# We love output :D
for m in out:
print(m)
For more info, you can consult this question.
Related Topics
Why Does Python Code Use Len() Function Instead of a Length Method
How to Use Angularjs with the Jinja2 Template Engine
List Comprehension VS Generator Expression's Weird Timeit Results
Convert Pandas Dataframe to Nested JSON
"Too Many Values to Unpack" Exception
Computing the Correlation Coefficient Between Two Multi-Dimensional Arrays
Python Unexpected Eof While Parsing
Finding Indices of Matches of One Array in Another Array
How to Flatten a Nested JSON Recursively, with Flatten_JSON
Argument 1 Has Unexpected Type 'Nonetype'
How to Add Both File and JSON Body in a Fastapi Post Request
Any Way to Modify Locals Dictionary
Tensorflow Only Running on 1/32 of the Training Data Provided
How to Switch Position of Two Items in a Python List
Returning the Product of a List