Do Python for Loops Work by Reference

Do Python for loops work by reference?

No, variables in Python are not pointers.

They refer to objects on a heap instead, and assigning to a variable doesn't change the referenced object, but the variable. Variables and objects are like labels tied to balloons; assignment reties the label to a different balloon instead.

See this previous answer of mine to explore that idea of balloons and labels a bit more.

That said, some object types implement specific in-place addition behaviour. If the object is mutable (the balloon itself can change), then an in-place add could be interpreted as a mutation instead of an assignment.

So, for integers, item += 1 is really the same as item = item + 1 because integers are immutable. You have to create a new integer object and tie the item label to that new object.

Lists on the other hand, are mutable and lst += [other, items] is implemented as a lst.__iadd__([other, items]) and that changes the lst balloon itself. An assignment still takes place, but it is a reassigment of the same object, as the .__iadd__() method simply returns self instead of a new object. We end up re-tying the label to the same balloon.

The loop simply gives you a reference to the next item in the list on each iteration. It does not let you change the original list itself (that's just another set of balloon labels); instead it gives you a new label to each of the items contained.

Looping over a list of objects in Python, reference/assignment works unexpectedly

The shortest answer is that python variables are like C pointers (if you are familiar with C), and your code is performing exactly as would be expected. To go through your questions in detail:

1. When you write x = 1 in python you are saying that the variable x should point to the bit of memory storing the value of 1. If you then write y = x you are saying y should point to the bit of memory storing the value of x, which is found by looking at the bit of memory pointed to by x. For all intents and purposes these two variables are the same now, and modifying the state of one will modify the state of the other. However if you then wrote x = 4, then you are asking for x to point to the memory storing the value of 4 and y is still pointing to it's previous value of 1.

   The way this affects your program is that when you write best_successor_state = successor_state you are asking best_successor_state to point to the value of successor_state which is found by looking at the value successor_state points to. So after this statement the two variables are effectively the same thing. Then when you reach the for successor_state in current_state.successor_states(): you are effectively calling successor_state = <SOME_OTHER_STATE>. Now best_successor_state is pointing to the value that successor_state used to have and successor_state is pointing to the new state; once again they are no longer equivalent objects.

2. What you are asking here is for successor_state and best_successor_state to be the same object at all times. The only way I see to do this is to replace for successor_state in current_state.successor_sates(): with somethings silly like for best_successor_state in current_state.successor_states():.

3. You assign a variable when ever you have something of the form x = y. You reference a variable whenever you are looking at it's value, so in x = y you are referencing y. Variable is used to mean a value that you can manipulate in a program, and I would definitely use that over label or name (you might say that "x" is the label for variable x, but that is getting somewhat into semantics).

Should the self reference be used for iteration variables in for loops inside classes?

The answer is generally "no". The for loop is assigning the variable to point to each element in the list. When it's done, it persists outside the for loop. The better way would be to keep element a local variable, and it will go away when the method ends. But if you assign it to self.element, that's equivalent to self.element = my_list[-1] when the loop ends. Then it will persist when the method exists, and will be accessible from other methods and holders of the class instance. That's usually not what you want.

Why does the loop iterate after deleting the reference to the iterable?

The loop isn't actually working with x; it's working with the return value of iter(x), which itself holds a reference to the same object as x. Deleting x doesn't affect that reference.

del x only decrements the reference count of the object referenced by x, and if that reference count reaches 0, the object is subject to garbage collection. It does not immediately destroy the object.

The first time through the loop, the name x is still defined. That name is removed, though, by del x, so on the second iteration, del x produces the NameError.

---

You can think of this for loop as being roughly equivalent to

x_iter = iter(x)
while True:
    try:
        i = next(x_iter)
    except StopIteration:
        del x_iter  # Clean up the namespace before exiting the loop
        break
    print(i)
    del x

(roughly, because unlike the name x_iter in this loop, the iterator used by the for loop as no Python-visible reference).

Python for loop initialization of reference to function

The problem can be shown more concisely as follows:

>>> rules = list([None, None])
>>> for a in range(2):
...   rules[a] = lambda t: a
... 
>>> rules[0](0)
1
>>> rules[0](1)
1
>>> rules[1](0)
1
>>> rules[1](1)
1

I think the problem is that the code always reflects the final value of a.

This is known as "late-binding closures" and is discussed in the Python Guide here.

One (rather ugly) way of getting round this is to create the new function each time by partially applying a function using the functools package. This "captures" the current value of a.

>>> from functools import partial
>>> for a in range(2):
...   def get(t,x): return x
...   rules[a] = partial(get,x=a)
... 
>>> rules[0](0)
0
>>> rules[0](1)
0
>>> rules[1](0)
1
>>> rules[1](1)
1

A simpler way of achieving the same effect:

>>> for a in range(2):
...   rules[a] = lambda t,a=a: a

As shown in the linked Python Guide, you can also use a list comprehension to simplify the code a little:

rules = [lambda t,a=a: a for a in range(2)]

How do I operate on the actual object, not a copy, in a python for loop?

Here ya go:

# Your for loop should be rewritten as follows:
for index in xrange(len(a)):
    a[index] += 1

Incidentally, item IS a reference to the item in a, but of course you can't assign a new value to an integer. For any mutable type, your code would work just fine:

>>> a = [[1], [2], [3], [4]]
>>> for item in a: item += [1]
>>> a
[[1,1], [2,1], [3,1], [4,1]]

when a python list iteration is and is not a reference

The loop variable d is always a reference to an element of the iterable object. The question is not really a matter of when or when isn't it a reference. It is about the assignment operation that you are performing with the loop.

In the first example, you are rebinding the original reference of an element in the object, with another reference to an empty string. This means you don't actually do anything to the value. You just assign a new reference to the symbol.

In the second example, you are performing an indexing operation and assigning a new reference to the value at that index. demo remains the same reference, and you are replacing a value in the container.

The assignment is really the equivalent of: demo.__setitem__(c, "")

a = 'foo'
id(a)  # 4313267976
a = 'bar'
id(a)  # 4313268016

l = ['foo']
id(l)  # 4328132552
l[0] = 'bar'
id(l)  # 4328132552

Notice how in the first example, the object id has changed. It is a reference to a new object. In the second one, we index into the list and replace a value in the container, yet the list remains the same object.

---

Related Topics