"Is" Operator Behaves Unexpectedly With Integers

is operator behaves unexpectedly with integers

Take a look at this:

>>> a = 256
>>> b = 256
>>> id(a)
9987148
>>> id(b)
9987148
>>> a = 257
>>> b = 257
>>> id(a)
11662816
>>> id(b)
11662828

Here's what I found in the Python 2 documentation, "Plain Integer Objects" (It's the same for Python 3):

The current implementation keeps an
array of integer objects for all
integers between -5 and 256, when you
create an int in that range you
actually just get back a reference to
the existing object. So it should be
possible to change the value of 1. I
suspect the behaviour of Python in
this case is undefined. :-)

The `is` operator behaves unexpectedly with non-cached integers

tl;dr:

As the reference manual states:

A block is a piece of Python program text that is executed as a unit.
The following are blocks: a module, a function body, and a class definition.
Each command typed interactively is a block.

This is why, in the case of a function, you have a single code block which contains a single object for the numeric literal
1000, so id(a) == id(b) will yield True.

In the second case, you have two distinct code objects each with their own different object for the literal 1000 so id(a) != id(b).

Take note that this behavior doesn't manifest with int literals only, you'll get similar results with, for example, float literals (see here).

Of course, comparing objects (except for explicit is None tests ) should always be done with the equality operator == and not is.

_{Everything stated here applies to the most popular implementation of Python, CPython. Other implementations might differ so no assumptions should be made when using them.}

Longer Answer:

To get a little clearer view and additionally verify this seemingly odd behaviour we can look directly in the code objects for each of these cases using the dis module.

For the function func:

Along with all other attributes, function objects also have a __code__ attribute that allows you to peek into the compiled bytecode for that function. Using dis.code_info we can get a nice pretty view of all stored attributes in a code object for a given function:

>>> print(dis.code_info(func))
Name:              func
Filename:          <stdin>
Argument count:    0
Kw-only arguments: 0
Number of locals:  2
Stack size:        2
Flags:             OPTIMIZED, NEWLOCALS, NOFREE
Constants:
   0: None
   1: 1000
Variable names:
   0: a
   1: b

We're only interested in the Constants entry for function func. In it, we can see that we have two values, None (always present) and 1000. We only have a single int instance that represents the constant 1000. This is the value that a and b are going to be assigned to when the function is invoked.

Accessing this value is easy via func.__code__.co_consts[1] and so, another way to view our a is b evaluation in the function would be like so:

>>> id(func.__code__.co_consts[1]) == id(func.__code__.co_consts[1])

Which, of course, will evaluate to True because we're referring to the same object.

For each interactive command:

As noted previously, each interactive command is interpreted as a single code block: parsed, compiled and evaluated independently.

We can get the code objects for each command via the compile built-in:

>>> com1 = compile("a=1000", filename="", mode="single")
>>> com2 = compile("b=1000", filename="", mode="single")

For each assignment statement, we will get a similar looking code object which looks like the following:

>>> print(dis.code_info(com1))
Name:              <module>
Filename:          
Argument count:    0
Kw-only arguments: 0
Number of locals:  0
Stack size:        1
Flags:             NOFREE
Constants:
   0: 1000
   1: None
Names:
   0: a

The same command for com2 looks the same but has a fundamental difference: each of the code objects com1 and com2 have different int instances representing the literal 1000. This is why, in this case, when we do a is b via the co_consts argument, we actually get:

>>> id(com1.co_consts[0]) == id(com2.co_consts[0])
False

Which agrees with what we actually got.

Different code objects, different contents.

Note: I was somewhat curious as to how exactly this happens in the source code and after digging through it I believe I finally found it.

During compilations phase the co_consts attribute is represented by a dictionary object. In compile.c we can actually see the initialization:

/* snippet for brevity */

u->u_lineno = 0;
u->u_col_offset = 0;
u->u_lineno_set = 0;
u->u_consts = PyDict_New();  

/* snippet for brevity */

During compilation this is checked for already existing constants. See @Raymond Hettinger's answer below for a bit more on this.

Caveats:

Chained statements will evaluate to an identity check of True
It should be more clear now why exactly the following evaluates to True:
```
 >>> a = 1000; b = 1000;
 >>> a is b
```
In this case, by chaining the two assignment commands together we tell the interpreter to compile these together. As in the case for the function object, only one object for the literal 1000 will be created resulting in a True value when evaluated.
Execution on a module level yields True again:
As previously mentioned, the reference manual states that:
... The following are blocks: a module ...
So the same premise applies: we will have a single code object (for the module) and so, as a result, single values stored for each different literal.
The same doesn't apply for mutable objects:

Meaning that unless we explicitly initialize to the same mutable object (for example with a = b = []), the identity of the objects will never be equal, for example:

    a = []; b = []
    a is b  # always evaluates to False

Again, in the documentation, this is specified:

after a = 1; b = 1, a and b may or may not refer to the same object with the value one, depending on the implementation, but after c = []; d = [], c and d are guaranteed to refer to two different, unique, newly created empty lists.

Why does the 'is' operator behave unexpectedly with arithmetically equal expressions

When you do something like :

(case-1)

a = 1000
b = a

or (case-2)

a = 1000
b = 1000

Python is smart enough to know before hand that even after execution you won't need new memory.

So, python just before execution makes b an alias of a in the first case.

The second case is bit different.
Python is a true object oriented language, the literal 1000 is treated as an object. (Intuitively you can think as 1000 to be name of a const object).

So in second case a and b are technically, both becoming alias of 1000

Now in your example:

a = 1000
b = 1000 + a - a
print (a == b)
print (a is b)

while assignment of b, python doesn't know before hand what is going to be the value of a. When I say before-hand I mean before any form of calculation being started. So python reserves a new memory location for band then saves the output of the operation in this new memory location.

It is also worth noting this:

4-1 is 3
True

In this case, python doesn't saves this line with 4-1 but processes it before compilation to be 3, for runtime optimisation.

is' operator behaves unexpectedly with floats

This has to do with how is works. It checks for references instead of value. It returns True if either argument is assigned to the same object.

In this case, they are different instances; float(0) and float(0) have the same value ==, but are distinct entities as far as Python is concerned. CPython implementation also caches integers as singleton objects in this range -> [x | x ∈ ℤ ∧ -5 ≤ x ≤ 256 ]:

>>> 0.0 is 0.0
True
>>> float(0) is float(0)  # Not the same reference, unique instances.
False

In this example we can demonstrate the integer caching principle:

>>> a = 256
>>> b = 256
>>> a is b
True
>>> a = 257
>>> b = 257
>>> a is b
False

Now, if floats are passed to float(), the float literal is simply returned (short-circuited), as in the same reference is used, as there's no need to instantiate a new float from an existing float:

>>> 0.0 is 0.0
True
>>> float(0.0) is float(0.0)
True

This can be demonstrated further by using int() also:

>>> int(256.0) is int(256.0)  # Same reference, cached.
True
>>> int(257.0) is int(257.0)  # Different references are returned, not cached.
False
>>> 257 is 257  # Same reference.
True
>>> 257.0 is 257.0  # Same reference. As @Martijn Pieters pointed out.
True

However, the results of is are also dependant on the scope it is being executed in (beyond the span of this question/explanation), please refer to user: @Jim's fantastic explanation on code objects. Even python's doc includes a section on this behavior:

5.9 Comparisons

[7]
Due to automatic garbage-collection, free lists, and the dynamic nature of descriptors, you may notice seemingly unusual behaviour in certain uses of the is operator, like those involving comparisons between instance methods, or constants. Check their documentation for more info.

Why does the float object behave differently with the is operator?

Mutable objects always create a new object, otherwise the data would be shared. There's not much to explain here, as if you append an item to an empty list, you don't want all of the empty lists to have that item.

Immutable objects behave in a completely different manner:

Strings get interned. If they are smaller than 20 alphanumeric characters, and are static (consts in the code, function names, etc), they get cached and are accessed from a special mapping reserved for these. It is to save memory but more importantly used to have a faster comparison. Python uses a lot of dictionary access operations under the hood which require string comparison. Being able to compare 2 strings like attribute or function names by comparing their memory address instead of the actual value, is a significant runtime improvement.
Booleans simply return the same object. Considering there are only 2 available, it makes no sense creating them again and again.
Small integers (from -5 to 256) by default, are also cached. These are used quite often, just about everywhere. Every time an integer is in that range, CPython simply returns the same object.

Floats however are not cached. Unlike integers, where the numbers 0-10 are extremely common, 1.0 isn't guaranteed to be more used than 2.0 or 0.1. That's why float() simply returns a new float. We could have optimized the empty float(), and we can check for speed benefits but it might not have made such a difference.

The confusion starts to arise when float(0.0) is float(0.0). Python has numerous optimizations built in:

First of all, consts are saved in each function's code object. 0.0 is 0.0 simply refers to the same object. It is a compile-time optimization.
Second of all, float(0.0) takes the 0.0 object, and since it's a float (which is immutable), it simply returns it. No need to create a new object if it's already a float.
Lastly, 1.0 + 1.0 is 2.0 will also work. The reason is that 1.0 + 1.0 is calculated on compile time and then references the same 2.0 object:
```
def test():
    return 1.0 + 1.0 is 2.0

dis.dis(test)
  2           0 LOAD_CONST               1 (2.0)
              2 LOAD_CONST               1 (2.0)
              4 IS_OP                    0
              6 RETURN_VALUE
```
As you can see, there is no addition operation. The function was compiled with the result pointing to the exact same constant object.

So while there is no float-specific optimization, 3 different generic optimizations are into play. The sum of them is what ultimately decides if it'll be the same object or not.

Confusion about Python operator 'is'

What you're doing when you're doing id(a) is id(list1[0]) is comparing the values returned by the id() function to check if they point to the same object or not.
Those values are different objects - EVEN IF THEY ARE THE SAME VALUE

Check this:

a = 2
ll = [2]

print(a is ll[0])
print(id(a), id(ll[0]))
print(id(a) is id(ll[0]))

Which gives:

True
140707131548528 140707131548528
False

Now why is the first result True? Because of interning - all ints between -5 & 256 are pre-created objects that are reused. So every 2 in python is actually the same object. But every 140707131548528 is different

Is the is Python operator reliable to test reference equality of mutable objects?

So long as you think some "is" behavior is "unexpected", your mental model falls short of reality ;-)

Your question is really about when Python guarantees to create a new object. And when it doesn't. For mutable objects, yes, a constructor (including a literal) yielding a mutable object always creates a new object. That's why:

>>> a = [0]
>>> b = [0]
>>> a is b

is always False. Python could have said that it's undefined whether each instance of [0] creates a new object, but it doesn't: it guarantees each instance always creates a new object. is behavior is a consequence of that, not a driver of that.

Similarly,

>>> a = set()
>>> b = set()
>>> a is b
False

is also guaranteed. Because set() returns a mutable object, it always guarantees to create a new such object.

But for immutable objects, it's not defined. For example, the result of this is not defined:

>>> a = frozenset()
>>> b = frozenset()
>>> a is b

frozenset() - like integer literals - returns an immutable object, and it's up to the implementation whether to return a new object or reuse an existing one. In this specific example, a is b is True, because the implementation du jour happens to reuse an empty frozenset. But, e.g., it just so happens that

>>> a = frozenset([3])
>>> b = frozenset([3])
>>> a is b
False

today. It could just as well return True tomorrow (although that's unlikely - while an empty frozenset is an easy-to-detect special case, it would be expensive to ensure uniqueness across all frozenset objects).

is operator in Python

is is not an equality operator. It checks to see if two variables refer to the same object. If you were to do this:

a = "12 34'
b = a

then a is b would be True, since they refer to the same object.

The cases you present are due to implementation details of the Python interpreter. Since strings are immutable, in some cases, creating two of the same string will yield references to the same object -- i.e., in your first case, the Python interpreter only creates a single copy of "1234", and a and b both refer to that object. In the second case, the interpreter creates two copies. This is due to the way the interpreter creates and handles strings, and, as an implementation detail, should not be relied upon.

Swapping variables and list elements in Python works unexpectedly

Simple assignments have no affect on the previous value of the target.

After

ls = [2222, 1111]

a = ls[0]

both a and the first element of ls are references to the integer 2222.

The assignment

a, ls[1] = ls[1], a

is effectively the same as

t = ls[1], a  # The tuple (1111, 2222)
a = t[0]  # Set a to t[0] == 1111
ls[1] = t[1]  # Set ls[1] to t[1] == 2222

At no point have you modified the first element of ls; you've only changed what a refers to and what the second element of ls refers to. You can see that a is now refers to 1111, since that's what the value of ls[1] was before ls[1] was modified.

>>> print(a)
1111

"Is" Operator Behaves Unexpectedly With Integers