Removing from a List While Iterating Over It

How to remove items from a list while iterating?

You can use a list comprehension to create a new list containing only the elements you don't want to remove:

somelist = [x for x in somelist if not determine(x)]

Or, by assigning to the slice somelist[:], you can mutate the existing list to contain only the items you want:

somelist[:] = [x for x in somelist if not determine(x)]

This approach could be useful if there are other references to somelist that need to reflect the changes.

Instead of a comprehension, you could also use itertools. In Python 2:

from itertools import ifilterfalse
somelist[:] = ifilterfalse(determine, somelist)

Or in Python 3:

from itertools import filterfalse
somelist[:] = filterfalse(determine, somelist)

Remove elements from collection while iterating

Let me give a few examples with some alternatives to avoid a ConcurrentModificationException.

Suppose we have the following collection of books

List<Book> books = new ArrayList<Book>();
books.add(new Book(new ISBN("0-201-63361-2")));
books.add(new Book(new ISBN("0-201-63361-3")));
books.add(new Book(new ISBN("0-201-63361-4")));

Collect and Remove

The first technique consists in collecting all the objects that we want to delete (e.g. using an enhanced for loop) and after we finish iterating, we remove all found objects.

ISBN isbn = new ISBN("0-201-63361-2");
List<Book> found = new ArrayList<Book>();
for(Book book : books){
    if(book.getIsbn().equals(isbn)){
        found.add(book);
    }
}
books.removeAll(found);

This is supposing that the operation you want to do is "delete".

If you want to "add" this approach would also work, but I would assume you would iterate over a different collection to determine what elements you want to add to a second collection and then issue an addAll method at the end.

Using ListIterator

If you are working with lists, another technique consists in using a ListIterator which has support for removal and addition of items during the iteration itself.

ListIterator<Book> iter = books.listIterator();
while(iter.hasNext()){
    if(iter.next().getIsbn().equals(isbn)){
        iter.remove();
    }
}

Again, I used the "remove" method in the example above which is what your question seemed to imply, but you may also use its add method to add new elements during iteration.

Using JDK >= 8

For those working with Java 8 or superior versions, there are a couple of other techniques you could use to take advantage of it.

You could use the new removeIf method in the Collection base class:

ISBN other = new ISBN("0-201-63361-2");
books.removeIf(b -> b.getIsbn().equals(other));

Or use the new stream API:

ISBN other = new ISBN("0-201-63361-2");
List<Book> filtered = books.stream()
                           .filter(b -> b.getIsbn().equals(other))
                           .collect(Collectors.toList());

In this last case, to filter elements out of a collection, you reassign the original reference to the filtered collection (i.e. books = filtered) or used the filtered collection to removeAll the found elements from the original collection (i.e. books.removeAll(filtered)).

Use Sublist or Subset

There are other alternatives as well. If the list is sorted, and you want to remove consecutive elements you can create a sublist and then clear it:

books.subList(0,5).clear();

Since the sublist is backed by the original list this would be an efficient way of removing this subcollection of elements.

Something similar could be achieved with sorted sets using NavigableSet.subSet method, or any of the slicing methods offered there.

Considerations:

What method you use might depend on what you are intending to do

• The collect and removeAl technique works with any Collection (Collection, List, Set, etc).
• The ListIterator technique obviously only works with lists, provided that their given ListIterator implementation offers support for add and remove operations.
• The Iterator approach would work with any type of collection, but it only supports remove operations.
• With the ListIterator/Iterator approach the obvious advantage is not having to copy anything since we remove as we iterate. So, this is very efficient.
• The JDK 8 streams example don't actually removed anything, but looked for the desired elements, and then we replaced the original collection reference with the new one, and let the old one be garbage collected. So, we iterate only once over the collection and that would be efficient.
• In the collect and removeAll approach the disadvantage is that we have to iterate twice. First we iterate in the foor-loop looking for an object that matches our removal criteria, and once we have found it, we ask to remove it from the original collection, which would imply a second iteration work to look for this item in order to remove it.
• I think it is worth mentioning that the remove method of the Iterator interface is marked as "optional" in Javadocs, which means that there could be Iterator implementations that throw UnsupportedOperationException if we invoke the remove method. As such, I'd say this approach is less safe than others if we cannot guarantee the iterator support for removal of elements.

Removing from a list while iterating over it

I debated answering this for a while, because similar questions have been asked many times here. But it's just unique enough to be given the benefit of the doubt. (Still, I won't object if others vote to close.) Here's a visual explanation of what is happening.

[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]       <-  b = 0; remove? no
 ^
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]       <-  b = 1; remove? yes
    ^
[0, 2, 3, 4, 5, 6, 7, 8, 9]          <-  b = 3; remove? no
       ^
[0, 2, 3, 4, 5, 6, 7, 8, 9]          <-  b = 4; remove? yes
          ^
[0, 2, 3, 5, 6, 7, 8, 9]             <-  b = 6; remove? no
             ^
[0, 2, 3, 5, 6, 7, 8, 9]             <-  b = 7; remove? yes
                ^
[0, 2, 3, 5, 6, 8, 9]                <-  b = 9; remove? no
                   ^

Since no one else has, I'll attempt to answer your other questions:

Why is no error given to indicate that underlying iterator is being modified?

To throw an error without prohibiting many perfectly valid loop constructions, Python would have to know a lot about what's going on, and it would probably have to get that information at runtime. All that information would take time to process. It would make Python a lot slower, in just the place where speed really counts -- a loop.

Have the mechanics changed from earlier versions of Python with respect to this behaviour?

In short, no. Or at least I highly doubt it, and certainly it has behaved this way since I learned Python (2.4). Frankly I would expect any straightforward implementation of a mutable sequence to behave in just this way. Anyone who knows better, please correct me. (Actually, a quick doc lookup confirms that the text that Mikola cited has been in the tutorial since version 1.4!)

Python: Removing list element while iterating over list

You could always iterate over a copy of the list, leaving you free to modify the original:

for item in list(somelist):
  ...
  somelist.remove(item)

Python: How to remove elements from list while iterating through it without skipping future iterations

You are right. You need an additional list. But there is an easier solution.

def print_numTXTs(fileList):

    counter = 0
    for file in list(fileList):
        if file.name[-4:] == ".txt":
            counter +=1
            if file.name == "a.txt":
                fileList.remove(file)
   

The secret is "list(fileList)". You creating an additional list and iterates over this.

Just as powerful are list compressions. In your example it should work like this. I have not tried now...only quickly written here.

fileList = [ file for file in fileList if file.name != "a.txt" ]

Removing items from a list while iterating over it

The best solution is not to maintain the currently playing song resp. the next song to play via an Iterator. Instead, you can create a specialized list which knows how to adapt this pointer on modifications.

Such a class could look like

class SongList extends AbstractList<Song> implements RandomAccess {
    final List<Song> backend = new ArrayList<>();
    int currentSong = -1;

    SongList() {}
    SongList(Collection<? extends Song> c) {
        backend.addAll(c);
    }
    // mandatory query methods

    @Override public int size() {
        return backend.size();
    }
    @Override public Song get(int index) {
        return backend.get(index);
    }

    // the "iterator"
    public Song nextSong() {
        if(++currentSong < size()) {
            return get(currentSong);
        }
        currentSong = -1;
        return null;
    }

    // modifying methods, which will adapt the pointer

    @Override public void add(int index, Song element) {
        backend.add(index, element);
        if(index <= currentSong) currentSong++;
    }
    @Override public Song remove(int index) {
        final Song removed = backend.remove(index);
        if(index <= currentSong) currentSong--;
        return removed;
    }

    @Override
    public boolean addAll(int index, Collection<? extends Song> c) {
        int old = size();
        backend.addAll(index, c);
        if(index <= currentSong) currentSong += size() - old;
        return true;
    }

    @Override protected void removeRange(int fromIndex, int toIndex) {
        backend.subList(fromIndex, toIndex).clear();
        if(fromIndex <= currentSong)
            currentSong = Math.max(fromIndex - 1, currentSong - toIndex + fromIndex);
    }

    // this will not change the pointer

    @Override public Song set(int index, Song element) {
        return backend.set(index, element);
    }

    // query methods overridden for performance

    @Override public boolean contains(Object o) {
        return backend.contains(o);
    }
    @Override public int indexOf(Object o) {
        return backend.indexOf(o);
    }
    @Override public Spliterator<Song> spliterator() {
        return backend.spliterator();
    }
    @Override public void forEach(Consumer<? super Song> action) {
        backend.forEach(action);
    }
    @Override public Object[] toArray() {
        return backend.toArray();
    }
    @Override public <T> T[] toArray(T[] a) {
        return backend.toArray(a);
    }
    @Override public String toString() {
        return backend.toString();
    }
}

AbstractList is specifically designed to provide the collection operations atop a few methods, so we only need to implement size() and get(int) to have a readable list and by providing add(int, Song), remove(int), and set(int, Song) we did already everything needed to support all modification operations. The other methods are only provided to improve the performance, the inherited methods would also work.

The list supports a single pointer to a current play position, which can be iterated via nextSong(). When reaching the end, it will return null and reset the pointer, so that the next query will start again. The add and remove methods will adapt the pointer such that an already played song won’t be played again (unless restarting the entire list).

set based modifications do not adapt the pointer, which implies that nothing meaningful will happen when you sort the list, some policies are imaginable, but at least when the list has duplicates, no perfect behavior exists. When comparing with other player software, no-one seems to expect perfect behavior when the list is turned up-side-down while playing. At least, there will never be an exception.

How to remove list elements in a for loop in Python?

You are not permitted to remove elements from the list while iterating over it using a for loop.

The best way to rewrite the code depends on what it is you're trying to do.

For example, your code is equivalent to:

for item in a:
    print(item)
a[:] = []

Alternatively, you could use a while loop:

while a:
    print(a.pop())

I'm trying to remove items if they match a condition. Then I go to next item.

You could copy every element that doesn't match the condition into a second list:

result = []
for item in a:
    if condition is False:
        result.append(item)
a = result

Alternatively, you could use filter or a list comprehension and assign the result back to a:

a = filter(lambda item:... , a)

or

a = [item for item in a if ...]

where ... stands for the condition that you need to check.

How to remove elements from a generic list while iterating over it?

Iterate your list in reverse with a for loop:

for (int i = safePendingList.Count - 1; i >= 0; i--)
{
    // some code
    // safePendingList.RemoveAt(i);
}

Example:

var list = new List<int>(Enumerable.Range(1, 10));
for (int i = list.Count - 1; i >= 0; i--)
{
    if (list[i] > 5)
        list.RemoveAt(i);
}
list.ForEach(i => Console.WriteLine(i));

Alternately, you can use the RemoveAll method with a predicate to test against:

safePendingList.RemoveAll(item => item.Value == someValue);

Here's a simplified example to demonstrate:

var list = new List<int>(Enumerable.Range(1, 10));
Console.WriteLine("Before:");
list.ForEach(i => Console.WriteLine(i));
list.RemoveAll(i => i > 5);
Console.WriteLine("After:");
list.ForEach(i => Console.WriteLine(i));

How can I remove an item from a list while iterating over it?

You still have a for loop inside your iterator loop. Replace it with this:

Process process = iter.next();

Delete from a list while iterating

You should use list comprehension and zip and instead of deleting elements from a , instead take elements in a whose b value is over 15. Example -

a[:] = [i for i,j in zip(a,b) if j >=15]

We are using a[:] on the left side, so that a list object gets mutated inplace. (This is different from a = <something> as the latter simply binds name a to a new list whereas former mutates the list inplace).

---

Demo -

>>> a = [1,2,3,4,5,6,7,8,9]
>>>
>>> b = [10,11,12,13,14,15,16,17,18]
>>> a[:] = [i for i,j in zip(a,b) if j >=15]
>>> a
[6, 7, 8, 9]

---

Related Topics