Why Does += Behave Unexpectedly on Lists

Why does creating a list of lists produce unexpected behavior?

When you do the following:

[[]]*n

You are first creating a list, then using the * operator with an int n. This takes whatever objects are in your list, and creates n- many repetitions of it.

But since in Python, explicit is better than implicit, you don't implicitly make a copy of those objects. Indeed, this is consistent with the semantics of Python.

Try to name a single case where Python implicitly makes a copy.

Furthermore, it is consistent with the addition on the list:

l = [1, [], 'a']

l2 = l + l + l

l[1].append('foo')

print(l2)

And the output:

[1, ['foo'], 'a', 1, ['foo'], 'a', 1, ['foo'], 'a']

Now, as noted in the comments, coming from C++ it makes sense that the above would be surprising, but if one is used to Python, the above is what one would expect.

On the other hand:

[[] for _ in range(5)]

Is a list comprehension. It is equivalent to:

lst = []
for _ in range(5):
    lst.append([])

Here, clearly, every time you are in the loop you create a new list. That is how literal syntax works.

As an aside, I almost never use the * operator on lists, except for one particular idiom I am fond of:

>>> x = list(range(1, 22))
>>> it_by_three = [iter(x)]*3
>>> for a,b,c in zip(*it_by_three):
...    print(a, b, c)
...
1 2 3
4 5 6
7 8 9
10 11 12
13 14 15
16 17 18
19 20 21

Why does Python list of lists behave differently depending on their declaration?

It's because of the first line:

>>> a = b = []
>>> a
[]
>>> b
[]
>>> a is b
True
>>> a = []
>>> b = []
>>> a is b
False
>>> 

With one line as in case 1, it contains the same object, so:

>>> a = b = []
>>> a.append(1)
>>> a
[1]
>>> b
[1]
>>> 

Which doesn't happen with two lines:

>>> a = []
>>> b = []
>>> a.append(1)
>>> a
[1]
>>> b
[]
>>> 

So simply because the first line of case 1 has a and b that are the exact same objects, unlike the second case's fist line, that are same values, but different id (id(a) == id(b) is the same as a is b).

Why does the 'is' operator behave unexpectedly with arithmetically equal expressions

When you do something like :

(case-1)

a = 1000
b = a

or (case-2)

a = 1000
b = 1000

Python is smart enough to know before hand that even after execution you won't need new memory.

So, python just before execution makes b an alias of a in the first case.

The second case is bit different.
Python is a true object oriented language, the literal 1000 is treated as an object. (Intuitively you can think as 1000 to be name of a const object).

So in second case a and b are technically, both becoming alias of 1000

Now in your example:

a = 1000
b = 1000 + a - a
print (a == b)
print (a is b)

while assignment of b, python doesn't know before hand what is going to be the value of a. When I say before-hand I mean before any form of calculation being started. So python reserves a new memory location for band then saves the output of the operation in this new memory location.

It is also worth noting this:

4-1 is 3
True

In this case, python doesn't saves this line with 4-1 but processes it before compilation to be 3, for runtime optimisation.

Weird behavior in python list concatenation

The reason behind this is that += and + calls two different methods of the class, __iadd__ method and __add__ method.

From an API perspective, iadd is supposed to be used for modifying mutable objects in place (returning the object which was mutated) whereas add should return a new instance of something. For immutable objects, both methods return a new instance, but iadd will put the new instance in the current namespace with the same name that the old instance had. This is why

i = 1
i += 1

seems to increment i. In reality, you get a new integer and assign it "on top of" i -- losing one reference to the old integer. In this case, i += 1 is exactly the same as i = i + 1. But, with most mutable objects, it's a different story:

As a concrete example:

a = [1, 2, 3]
b = a
b += [1, 2, 3]
print a  #[1, 2, 3, 1, 2, 3]
print b  #[1, 2, 3, 1, 2, 3]

compared to:

a = [1, 2, 3]
b = a
b = b + [1, 2, 3]
print a #[1, 2, 3]
print b #[1, 2, 3, 1, 2, 3]

notice how in the first example, since b and a reference the same object, when I use += on b, it actually changes b (and a sees that change too -- After all, it's referencing the same list). In the second case however, when I do b = b + [1, 2, 3], this takes the list that b is referencing and concatenates it with a new list [1, 2, 3]. It then stores the concatenated list in the current namespace as b -- With no regard for what b was the line before.

Unexpected list behavior in Python

The following:

list_reversed = list 

makes the two variables refer to the same list. When you change one, they both change.

To make a copy, use

list_reversed = list[:]

Better still, use the builtin function instead of writing your own:

list_reversed = reversed(list)

P.S. I'd recommend against using list as a variable name, since it shadows the builtin.

---

Related Topics