Weird Behavior: Lambda Inside List Comprehension

Weird behavior: Lambda inside list comprehension

To make the lambdas remember the value of m, you could use an argument with a default value:

[x() for x in [lambda m=m: m for m in [1,2,3]]]
# [1, 2, 3]

This works because default values are set once, at definition time. Each lambda now uses its own default value of m instead of looking for m's value in an outer scope at lambda execution time.

Weird lambda behaviour in list comprehension

lambdas bind variables themselves, not the values that they had. i is changed to 2 at the end of the list comprehension, so all the lambdas refer to i at that point, and thus refer to 2.

To avoid this, you can use the default argument trick:

[lambda x,i=i:i for i in range(3)]

This binds the value of i in the default argument (which is evaluated at function definition time).

Strange behavior with nested list comprehension

Is sl defined elsewhere in your code? Perhaps as 5? As written, your first example should not run, and does not run for me in Python 3.6. The correct way to write it would be

[e for sl in [1,[2,3],4,5] for e in [sl]]

Note that here sl is defined before it is used.

Edit:

Python reads list comprehensions left to right. When it gets to for e in [sl], it evaluates the expression [sl] based on what it already knows, without reading the rest of the line. You list comprehension is then something like

[e for e in [5] for sl in [1,[2,3],4,5]]

As there are four sl in [[1,[2,3],4,5]], you get 5 four times in the resultant list.

When writing list comprehensions, it's natural to write them from smallest to biggest

e for e in x for x in y for y in z #wrong

but you should actually write them from right to left, so that the interpreter recognizes the identifiers that you use in the nested comprehensions

e for y in z for x in y for e in x

This is no different from regular for loops:

for e in x:
    for x in y:
        for y in z:
            print(e)

is pretty obviously wrong, and list comprehensions are no different.

In Python, why do lambdas in list comprehensions overwrite themselves in retrospect?

The lambda didn't overwrite itself, it is i that was overwritten. This is a common mistake regarding variable scopes. Try this:

[lambda a, i=i: i(a) for i in (f,g)][0](0)

(the difference is binding the value of i at the time the lambda is created)

See also:

• Weird behavior: Lambda inside list comprehension
• What is “lambda binding” in Python?
• Python lambda's binding to local values

List of lambda functions does weird things

The lambda references the global variable i, so after the for loop, i==3, computing X[3]**2:

X = [1,2,3,4]
L = []

for i in range(len(X)):
   L.append(lambda x: X[i]**x)

for f in L:
    print(f(2))

Output:

16
16
16
16

A way to fix is to capture the current value of global i as a local parameter i when the function is defined:

X = [1,2,3,4]
L = []

for i in range(len(X)):
   L.append(lambda x, i=i: X[i]**x)  # capture i as a parameter

for f in L:
    print(f(2))

Output:

1
4
9
16

strange behavior with lamba: getattr(obj, x) inside a list

This is a very common problem with lambdas. Ultimately, the variable x is looked up when the function is called, not when it is created. As such, at the end of your loop, the value of x is 'prop4' and all your lambdas will give you the same thing.

The commonly proposed fix is to use a default argument in your lambda. It gets evaluated when the function is created.

lambda x=x: getattr(obj,x)

Weird lambda behaviour in loops

The function lambda: el used in loop_one refers to a variable el which is not defined in the local scope. Therefore, Python looks for it next in the enclosing scope of the other lambda:

lambda seq: [lambda: el for el in seq]

in accordance with the so-called LEGB rule.

By the time lambda: el is called, this enclosing lambda has (of course) already been called and the list comprehension has been evaluated. The el used in the list comprehension is a local variable in this enclosing lambda. Its value is the one returned when Python looks for the value of el in lambda: el. That value for el is the same for all the different lambda: el functions in the list comprehension: it is the last value assigned to el in the for el in seq loop. Thus, el is always 'spam', the last value in seq.

---

You've already found one workaround, to use a closure such as your loop_two. Another way is to define el as a local variable with a default value:

loop_one = lambda seq: [lambda el=el: el for el in seq]

Behaviour I don't understand in Python mixings comprehension lists and lambda-functions

The problem is that you are only ever evaluating ev at the time you call the function. Thus, it's using whatever value ev has only when you start the print statement. Of course, by that time ev has the value of the last function in the list.

This is no different than if you had done this:

funcs = [lambda x: ev.eval(x+0.1),
         lambda x: ev.eval(x+0.1),
         lambda x: ev.eval(x+0.1)]

Notice how they all use ev, and they will all use the same ev at the time you run the functions.

To do what you want, you need to bind ev to its current value in the list comprehension at the time you define the comprehension, which you can do by passing in the value through the lambda arguments:

funcs = [lambda x, ev=ev: ev.eval(x+0.1) for ev in (ev1, ev2, ev3)]

However, I strongly suggest you do not do this. As you have just experienced, such code is very hard to understand and debug. You win no points for cramming as much functionality into a single line as possible.

The technical term for this is a closure. For more information, check out this question on stackoverflow:
Why aren't python nested functions called closures?

Lambdas from a list comprehension are returning a lambda when called

In Python 2 list comprehension 'leaks' the variables to outer scope:

>>> [i for i in xrange(3)]
[0, 1, 2]
>>> i
2

Note that the behavior is different on Python 3:

>>> [i for i in range(3)]
[0, 1, 2]
>>> i
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'i' is not defined

When you define lambda it's bound to variable i, not its' current value as your second example shows.
Now when you assign new value to i the lambda will return whatever is the current value:

>>> a = [lambda: i for i in range(5)]
>>> a[0]()
4
>>> i = 'foobar'
>>> a[0]()
'foobar'

Since the value of i within the loop is the lambda itself you'll get it as a return value:

>>> i = a[0]
>>> i()
<function <lambda> at 0x01D689F0>
>>> i()()()()
<function <lambda> at 0x01D689F0>

UPDATE: Example on Python 2.7:

Python 2.7.6 (default, Jun 22 2015, 17:58:13) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [lambda: i for i in range(5)]
>>> for i in a:
...     print i()
... 
<function <lambda> at 0x7f1eae7f15f0>
<function <lambda> at 0x7f1eae7f1668>
<function <lambda> at 0x7f1eae7f16e0>
<function <lambda> at 0x7f1eae7f1758>
<function <lambda> at 0x7f1eae7f17d0>

Same on Python 3.4:

Python 3.4.3 (default, Oct 14 2015, 20:28:29) 
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a = [lambda: i for i in range(5)]
>>> for i in a:
...     print(i())
... 
4
4
4
4
4

For details about the change regarding the variable scope with list comprehension see Guido's blogpost from 2010.

We also made another change in Python 3, to improve equivalence between list comprehensions and generator expressions. In Python 2, the list comprehension "leaks" the loop control variable into the surrounding scope:

x = 'before'
a = [x for x in 1, 2, 3]
print x # this prints '3', not 'before'

However, in Python 3, we decided to fix the "dirty little secret" of list comprehensions by using the same implementation strategy as for generator expressions. Thus, in Python 3, the above example (after modification to use print(x) :-) will print 'before', proving that the 'x' in the list comprehension temporarily shadows but does not override the 'x' in the surrounding scope.

Lambda function in list comprehensions

The first one creates a single lambda function and calls it ten times.

The second one doesn't call the function. It creates 10 different lambda functions. It puts all of those in a list. To make it equivalent to the first you need:

[(lambda x: x*x)(x) for x in range(10)]

Or better yet:

[x*x for x in range(10)]

---

Related Topics