List Comprehension Rebinds Names Even After Scope of Comprehension. Is This Right

List comprehension rebinds names even after scope of comprehension. Is this right?

List comprehensions leak the loop control variable in Python 2 but not in Python 3. Here's Guido van Rossum (creator of Python) explaining the history behind this:

We also made another change in Python
3, to improve equivalence between list
comprehensions and generator
expressions. In Python 2, the list
comprehension "leaks" the loop control
variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original
implementation of list comprehensions;
it was one of Python's "dirty little
secrets" for years. It started out as
an intentional compromise to make list
comprehensions blindingly fast, and
while it was not a common pitfall for
beginners, it definitely stung people
occasionally. For generator
expressions we could not do this.
Generator expressions are implemented
using generators, whose execution
requires a separate execution frame.
Thus, generator expressions
(especially if they iterate over a
short sequence) were less efficient
than list comprehensions.

However, in Python 3, we decided to
fix the "dirty little secret" of list
comprehensions by using the same
implementation strategy as for
generator expressions. Thus, in Python
3, the above example (after
modification to use print(x) :-) will
print 'before', proving that the 'x'
in the list comprehension temporarily
shadows but does not override the 'x'
in the surrounding scope.

Why list comprehension can affect the value of an outer variable?

I think you're using Python 2.7, which leaks the variables defined in the comprehension. On the last iteration the i in the loop is equal to 4, and the name i from the comprehensionthen shadows the global name once control leaves the comprehension.

Python 3 doesn't do that anymore, though, so you should update.

Python 2.7 list comprehension leaks variable name

There is nothing wrong. Using a list comprehension adds its variable to the local scope, just as a for loop would. When used inside a class definition, said local scope is used to initialize the class attributes. You have to del the name s if you don't want it in your class.

class MyClass:
  some_strings = [s for s in ('Foo','Bar')]
  del s

Note that in Python 3 (tested with 3.4), the list comprehension will not add its variable to the local scope.

Scope of class variable with list comprehension

This isn't so much about the variable scope in list comprehensions as about the way classes work. In Python 3 (but not in Python 2!), list comprehensions don't affect the scope around them:

>>> [i for i in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> i
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined
>>> i = 0
>>> [i for i in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> i
0

However, when you do that in a class, it won't look for b among the class attributes the way it would in the local scope of a module or function. To do what you are trying to do, use the @property decorator:

>>> class a:
...   s = 'python'
...   b = 'py'
...   @property
...   def c(self):
...     return [x for x in self.s if x in self.b]
... 
>>> A = a()
>>> A.c
['p', 'y']

Also, remember that strings are iterable too (they're just lists of their component characters), so no need to explicitly make b a list.

Why the list comprehension variable is accessible after the operation is done?

No, it's not a memory leak as that term is usually defined. In Python 2.x, a list comprehension is not a separate scope, so a variable you use in a list comprehension is in the scope of the function that contains it. You can easily see this in action by setting k before the list comprehension; the listcomp will clobber it.

Because a valid reference exists, the object k points to is (properly) not garbage collected.

In Python 3.x, this was changed; all comprehensions create their own scopes and do not "leak" into the enclosing scope.

In Python 2.x, generator expressions do have their own scope, however, so if you want that behavior, just write it like this:

m = list(k**2 for k in range(7))

Python weird behavior in list comprehension

Python 2

The n variable used in the list comprehension is the same n as is passed in.

The comprehension sets it to 1, 2, and then finally 3.

Instead, change it to

arr = [0 for _ in range(n)]

or (surprisingly!)

arr = list(0 for n in range(n))

Python 3

This has been fixed.

From the BDFL himself:

We also made another change in Python 3, to improve equivalence
between list comprehensions and generator expressions. In Python 2,
the list comprehension "leaks" the loop control variable into the
surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original implementation of list
comprehensions; it was one of Python's "dirty little secrets" for
years. It started out as an intentional compromise to make list
comprehensions blindingly fast, and while it was not a common pitfall
for beginners, it definitely stung people occasionally. For generator
expressions we could not do this. Generator expressions are
implemented using generators, whose execution requires a separate
execution frame...

However, in Python 3, we decided to fix the "dirty little secret" of
list comprehensions by using the same implementation strategy as for
generator expressions. Thus, in Python 3, the above example (after
modification to use print(x) :-) will print 'before'.

Why do list comprehensions write to the loop variable, but generators don't?

Python’s creator, Guido van Rossum, mentions this when he wrote about generator expressions that were uniformly built into Python 3: (emphasis mine)

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension "leaks" the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

This was an artifact of the original implementation of list comprehensions; it was one of Python's "dirty little secrets" for years. It started out as an intentional compromise to make list comprehensions blindingly fast, and while it was not a common pitfall for beginners, it definitely stung people occasionally. For generator expressions we could not do this. Generator expressions are implemented using generators, whose execution requires a separate execution frame. Thus, generator expressions (especially if they iterate over a short sequence) were less efficient than list comprehensions.

However, in Python 3, we decided to fix the "dirty little secret" of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) :-) will print 'before', proving that the 'x' in the list comprehension temporarily shadows but does not override the 'x' in the surrounding scope.

So in Python 3 you won’t see this happen anymore.

Interestingly, dict comprehensions in Python 2 don’t do this either; this is mostly because dict comprehensions were backported from Python 3 and as such already had that fix in them.

There are some other questions that cover this topic too, but I’m sure you have already seen those when you searched for the topic, right? ;)

• Python list comprehension rebind names even after scope of comprehension. Is this right?
• Why the list comprehension variable is accessible after the operation is done?

Python 3 2-D list initialization breaks in Python 2

You are using i twice within the same for loop, once in the list-comprehension and once as a run variable for the outer for-loop. As detailed in this post, variables used in a list comprehension "leak" into the surrounding scope in python 2x. So your for loop processes like this:

1. Set i=0, from the for loop:

• List comprehension generates list of -1's and sets i=2

• myList[i] = is actually myList[2] =

2. Set i=1, from the for loop:

• List comprehension generates list of -1's and sets i=2

• myList[i] = is actually myList[2] =

3. Set i=2, from the for loop:

• List comprehension generates list of -1's and sets i=2

• myList[i] = is actually myList[2] =

Which generates the list [0, 0, [-1, -1, -1]], because we only ever changed myList[2]

The simplest fix is to use different variables, i.e j in one place:

from __future__ import print_function

ROW = 3
COL = 3

myList = [0 for i in range(ROW)]

for i in range(len(myList)):
    #Use j for the list comprehension
    myList[i] = [-1 for j in range(COL)]

print('myList2 =', myList)

---

Related Topics