Different Behaviour for List._Iadd_ and List._Add_

Different behaviour for list.__iadd__ and list.__add__

__iadd__ mutates the list, whereas __add__ returns a new list, as demonstrated.

An expression of x += y first tries to call __iadd__ and, failing that, calls __add__ followed an assignment (see Sven's comment for a minor correction). Since list has __iadd__ then it does this little bit of mutation magic.

Python: Difference between list.extend and list.__iadd__

+= gives an object the opportunity to alter the object in-place. But this depends on the type of x, it is not a given that the object is altered in place.

As such, += still needs to re-assign to x; either x.__iadd__() returns x, or a new object is returned; x += something is really translated to:

x = x.__iadd__(something)

Because += includes an assignment, x is marked as a local in g().

x.extend() on the other hand, is not an assignment. The programmer has decided that x is always an object with an .extend() method and uses it directly. Python sees no assignment and x is marked as a global.

Weird behavior in python list concatenation

The reason behind this is that += and + calls two different methods of the class, __iadd__ method and __add__ method.

From an API perspective, iadd is supposed to be used for modifying mutable objects in place (returning the object which was mutated) whereas add should return a new instance of something. For immutable objects, both methods return a new instance, but iadd will put the new instance in the current namespace with the same name that the old instance had. This is why

i = 1
i += 1

seems to increment i. In reality, you get a new integer and assign it "on top of" i -- losing one reference to the old integer. In this case, i += 1 is exactly the same as i = i + 1. But, with most mutable objects, it's a different story:

As a concrete example:

a = [1, 2, 3]
b = a
b += [1, 2, 3]
print a  #[1, 2, 3, 1, 2, 3]
print b  #[1, 2, 3, 1, 2, 3]

compared to:

a = [1, 2, 3]
b = a
b = b + [1, 2, 3]
print a #[1, 2, 3]
print b #[1, 2, 3, 1, 2, 3]

notice how in the first example, since b and a reference the same object, when I use += on b, it actually changes b (and a sees that change too -- After all, it's referencing the same list). In the second case however, when I do b = b + [1, 2, 3], this takes the list that b is referencing and concatenates it with a new list [1, 2, 3]. It then stores the concatenated list in the current namespace as b -- With no regard for what b was the line before.

Python unexpected behaviour with list.append()

I believe the issue is that model.run is returning a reference to mutable state within the model which is updated on successive run calls -- that's why there's a difference between what you print immediately after the run and what ends up in the list after additional calls. I.e. the value is correct at the time you call append, but it changes after the fact. When you call model.run(x[2]), it's modifying and returning the list that it returned when you called model.run(x[1]), and at some point it also modifies the list that was returned for x[0]. (This is not good behavior IMO -- if model is from an external library, hopefully it's at least documented in the API that you should not keep references to the return value of run! Otherwise it's just plain diabolical.)

To work around this problem so that you can keep each result as it was originally returned, make a copy of each result as you get it:

out_values = [model.run(value).copy() for value in x]

Some strange behavior Python list and dict

That's a normal behaviour. Python uses references to store elements.
When you do r.append(t) python will store t in r. If you modify t later, t in r will be also modified because it's the same object.

If you want to make t independant from the value stored in r you have to copy it. Look at the copy module.

map vs list; why different behaviour?

The answer is very simple: map is a lazy function in Python 3, it returns an iterable object (in Python 2 it returns a list). Let me add some output to your example:

In [6]: nums = [1, 2, 3]

In [7]: for x in [4, 5, 6]:
   ...:     nums = map(lambda n: n if x % 2 else n + 10, nums)
   ...:     print(x)
   ...:     print(nums)
   ...:     
4
<map object at 0x7ff5e5da6320>
5
<map object at 0x7ff5e5da63c8>
6
<map object at 0x7ff5e5da6400>

In [8]: print(x)
6

In [9]: list(nums)
Out[9]: [31, 32, 33]

Note the In[8] - the value of x is 6. We could also transform the lambda function, passed to map in order to track the value of x:

In [10]: nums = [1, 2, 3]

In [11]: for x in [4, 5, 6]:
   ....:     nums = map(lambda n: print(x) or (n if x % 2 else n + 10), nums)
   ....:     

In [12]: list(nums)
6
6
6
6
6
6
6
6
6
Out[12]: [31, 32, 33]

Because map is lazy, it evaluates when list is being called. However, the value of x is 6 and that is why it produces confusing output. Evaluating nums inside the loop produces expected output.

In [13]: nums = [1, 2, 3]

In [14]: for x in [4, 5, 6]:
   ....:     nums = map(lambda n: print(x) or (n if x % 2 else n + 10), nums)
   ....:     nums = list(nums)
   ....:     
4
4
4
5
5
5
6
6
6

In [15]: nums
Out[15]: [21, 22, 23]

Is the behaviour of Python's list += iterable documented anywhere?

From Guido van Rossum:

It works the same way as .extend() except that it also returns self. I
can't find docs explaining this. :-(

Here is the relevant source code taken from listobject.c:

list_inplace_concat(PyListObject *self, PyObject *other)
{
     PyObject *result;

     result = listextend(self, other);
     if (result == NULL)
         return result;
     Py_DECREF(result);
     Py_INCREF(self);
     return (PyObject *)self;
}

I've raised a bug report to have the documentation fixed: http://bugs.python.org/issue16701

---

Related Topics