How Unique Is Python's Id()

How unique is Python's id()?

Yes, CPython re-uses id() values. Do not count on these being unique in a Python program.

This is clearly documented:

Return the “identity” of an object. This is an integer which is guaranteed to be unique and constant for this object during its lifetime. Two objects with non-overlapping lifetimes may have the same id() value.

Bold emphasis mine. The id is unique only as long as an object is alive. Objects that have no references left to them are removed from memory, allowing the id() value to be re-used for another object, hence the non-overlapping lifetimes wording.

Note that this applies to CPython only, the standard implementation provided by python.org. There are other Python implementations, such as IronPython, Jython and PyPy, that make their own choices about how to implement id(), because they each can make distinct choices on how to handle memory and object lifetimes.

To address your specific questions:

1. In CPython, id() is the memory address. New objects will be slotted into the next available memory space, so if a specific memory address has enough space to hold the next new object, the memory address will be reused. You can see this in the interpreter when creating new objects that are the same size:

   >>> id(1234)
   4546982768
   >>> id(4321)
   4546982768
   

   The 1234 literal creates a new integer object, for which id() produces a numeric value. As there are no further references to the int value, it is removed from memory again. But executing the same expression again with a different integer literal, and chances are you'll see the same id() value (a garbage collection run breaking cyclic references could free up more memory, so you could also not see the same id() again.

   So it's not random, but in CPython it is a function of the memory allocation algorithms.

2. If you need to check specific objects, keep your own reference to it. That can be a weakref weak reference if all you need to assure is that the object is still 'alive'.

   For example, recording an object reference first, then later checking it:

   import weakref
   
   # record
   object_ref = weakref.ref(some_object)
   
   # check if it's the same object still
   some_other_reference is object_ref()   # only true if they are the same object
   

   The weak reference won't keep the object alive, but if it is alive then the object_ref() will return it (it'll return None otherwise).

   You could use such a mechanism to generate really unique identifiers, see below.

3. All you have to do to 'destroy' an object is to remove all references to it. Variables (local and global) are references. So are attributes on other objects, and entries in containers such as lists, tuples, dictionaries, sets, etc.

   The moment all references to an object are gone, the reference count on the object drops to 0 and it is deleted, there and then.

   Garbage collection only is needed to break cyclic references, objects that reference one another only, with no further references to the cycle. Because such a cycle will never reach a reference count of 0 without help, the garbage collector periodically checks for such cycles and breaks one of the references to help clear those objects from memory.

   So you can cause any object to be deleted from memory (freed), by removing all references to it. How you achieve that depends on how the object is referenced. You can ask the interpreter to tell you what objects are referencing a given object with the gc.get_referrers() function, but take into account that doesn't give you variable names. It gives you objects, such as the dictionary object that is the __dict__ attribute of a module that references the object as a global, etc. For code fully under your control, at most use gc.get_referrers() as a tool to remind yourself what places the object is referenced from as you write the code to remove those.

If you must have unique identifiers for the lifetime of the Python application, you'd have to implement your own facility. If your objects are hashable and support weak references, then you could just use a WeakKeyDictionary instance to associate arbitrary objects with UUIDs:

from weakref import WeakKeyDictionary
from collections import defaultdict
from uuid import uuid4

class UniqueIdMap(WeakKeyDictionary):
    def __init__(self, dict=None):
        super().__init__(self)
        # replace data with a defaultdict to generate uuids
        self.data = defaultdict(uuid4)
        if dict is not None:
            self.update(dict)

uniqueidmap = UniqueIdMap()

def uniqueid(obj):
    """Produce a unique integer id for the object.

    Object must me *hashable*. Id is a UUID and should be unique
    across Python invocations.

    """
    return uniqueidmap[obj].int

This still produces integers, but as they are UUIDs they are not quite guaranteed to be unique, but the likelihood you'll ever encounter the same ID during your lifetime are smaller than being hit by a meteorite. See How unique is UUID?

This then gives you unique ids even for objects with non-overlapping lifetimes:

>>> class Foo:
...     pass
...
>>> id(Foo())
4547149104
>>> id(Foo())  # memory address reused
4547149104
>>> uniqueid(Foo())
151797163173960170410969562162860139237
>>> uniqueid(Foo())  # but you still get a unique UUID
188632072566395632221804340107821543671

understanding python id() uniqueness

Ids are guaranteed to be unique for the lifetime of the object. If an object gets deleted, a new object can acquire the same id. CPython will delete items immediately when their refcount drops to zero. The garbage collector is only needed to break up reference cycles.

CPython may also cache and re-use certain immutable objects like small integers and strings defined by literals that are valid identifiers. This is an implementation detail that you should not rely upon. It is generally considered improper to use is checks on such objects.

There are certain exceptions to this rule, for example, using an is check on possibly-interned strings as an optimization before comparing them with the normal == operator is fine. The dict builtin uses this strategy for lookups to make them faster for identifiers.

a is b or a == b  # This is OK

If the string happens to be interned, then the above can return true with a simple id comparison instead of a slower character-by-character comparison, but it still returns true if and only if a == b (because if a is b then a == b must also be true). However, a good implementation of .__eq__() would already do an is check internally, so at best you would only avoid the overhead of calling the .__eq__().

---

Thanks for the answer, would you elaborate around the uniqueness for user-defined objects, are they always unique?

The id of any object (be it user-defined or not) is unique for the lifetime of the object. It's important to distinguish objects from variables. It's possible to have two or more variables refer to the same object.

>>> a = object()
>>> b = a
>>> c = object()
>>> a is b
True
>>> a is c
False

Caching optimizations mean that you are not always guaranteed to get a new object in cases where one might naiively think one should, but this does not in any way violate the uniqueness guarantee of IDs. Builtin types like int and str may have some caching optimizations, but they follow exactly the same rules: If they are live at the same time, and their IDs are the same, then they are the same object.

Caching is not unique to builtin types. You can implement caching for your own objects.

>>> def the_one(it=object()):
...     return it
...
>>> the_one() is the_one()
True

Even user-defined classes can cache instances. For example, this class only makes one instance of itself.

>>> class TheOne:
...     _the_one = None
...     def __new__(cls):
...         if not cls._the_one:
...             cls._the_one = super().__new__(cls)
...         return cls._the_one
...
>>> TheOne() is TheOne()  # There can be only one TheOne.
True
>>> id(TheOne()) == id(TheOne())  # This is what an is-check does.
True

Note that each construction expression evaluates to an object with the same id as the other. But this id is unique to the object. Both expressions reference the same object, so of course they have the same id.

The above class only keeps one instance, but you could also cache some other number. Perhaps recently used instances, or those configured in a way you expect to be common (as ints do), etc.

Why is the id of a Python class not unique when called quickly?

The id of an object is only guaranteed to be unique during that object's lifetime, not over the entire lifetime of a program. The two someClass objects you create only exist for the duration of the call to print - after that, they are available for garbage collection (and, in CPython, deallocated immediately). Since their lifetimes don't overlap, it is valid for them to share an id.

It is also unsuprising in this case, because of a combination of two CPython implementation details: first, it does garbage collection by reference counting (with some extra magic to avoid problems with circular references), and second, the id of an object is related to the value of the underlying pointer for the variable (ie, its memory location). So, the first object, which was the most recent object allocated, is immediately freed - it isn't too surprising that the next object allocated will end up in the same spot (although this potentially also depends on details of how the interpreter was compiled).

If you are relying on several objects having distinct ids, you might keep them around - say, in a list - so that their lifetimes overlap. Otherwise, you might implement a class-specific id that has different guarantees - eg:

class SomeClass:
    next_id = 0

    def __init__(self):
         self.id = SomeClass.nextid
         SomeClass.nextid += 1

Is there an object unique identifier in Python

id(x)

will do the trick for you. But I'm curious, what's wrong about the set of objects (which does combine objects by value)?

For your particular problem I would probably keep the set of ids or of wrapper objects. A wrapper object will contain one reference and compare by x==y <==> x.ref is y.ref.

It's also worth noting that Python objects have a hash function as well. This function is necessary to put an object into a set or dictionary. It is supposed to sometimes collide for different objects, though good implementations of hash try to make it less likely.

What is the id( ) function used for?

Your post asks several questions:

What is the number returned from the function?

It is "an integer (or long integer) which is guaranteed to be unique and constant for this object during its lifetime." (Python Standard Library - Built-in Functions) A unique number. Nothing more, and nothing less. Think of it as a social-security number or employee id number for Python objects.

Is it the same with memory addresses in C?

Conceptually, yes, in that they are both guaranteed to be unique in their universe during their lifetime. And in one particular implementation of Python, it actually is the memory address of the corresponding C object.

If yes, why doesn't the number increase instantly by the size of the data type (I assume that it would be int)?

Because a list is not an array, and a list element is a reference, not an object.

When do we really use id( ) function?

Hardly ever. You can test if two references are the same by comparing their ids, but the is operator has always been the recommended way of doing that. id( ) is only really useful in debugging situations.

Creating a unique id in a python dataclass

Use a default factory instead of just a default. This allows to define a call to get the next id on each instantiation.

A simple means to get a callable that counts up is to use count().__next__, the equivalent of calling next(...) on a count instance.¹

The common "no explicit ctor" libraries attr and dataclasses both support this:

from itertools import count
from dataclasses import dataclass, field

@dataclass
class C:
    identifier: int = field(default_factory=count().__next__)

import attr

@attr.s
class C:
    identifier: int = attr.field(factory=count().__next__)

To always use the automatically generated value and prevent passing one in as a parameter, use init=False.

@dataclass
class C:
    identifier: int = field(default_factory=count().__next__, init=False)

---

¹ If one wants to avoid explicitly addressing magic methods, one can use a closure over a count. For example, factory=lambda counter=count(): next(counter).

---

Related Topics