Special (Magic) Methods in Python

Special (magic) methods in Python

At the python level, most of them are documented in the language reference. At the C level, you can find it under the object protocol section (strictly speaking, you only have a subset here, though).

is there a magic method for sorted() in Python?

You should definitely read the official documentation of how to emulate container types. Basically a class supposed to work as a container (list, dict etc.) needs to implement methods to set or get members __getitem__(), __setitem__() and iterate over items __iter__() and to get the number of items - method __len__(). This is the minimum. But you can also add the ability to delete items and other operations.

The behaviour of sorted() built-in function is to iterate over elements of your container and compare them using methods you mentioned __cmp__(), __ge__(), __le__() which should be defined for items and not the container as you know already. Then a new list instance is created with items sorted and this new instance is returned. You can pass it to the constructor of your custom container then or you can wriap sorted() with a custom function which will return the desired class instance.

Should I use python magic methods directly?

I intended to show some benefits of not using magic methods directly:

1- Readability:

Using built-in functions like len() is much more readable than its relevant magic/special method __len__(). Imagine a source code full of only magic methods instead of built-in function... thousands of underscores...

---

2- Comparison operators:

class C:
    def __lt__(self, other):
        print('__lt__ called')

class D:
    pass

c = C()
d = D()

d > c
d.__gt__(c)

I haven't implemented __gt__ for neither of those classes, but in d > c when Python sees that class D doesn't have __gt__, it checks to see if class C implements __lt__. It does, so we get '__lt__ called' in output which isn't the case with d.__gt__(c).

---

3- Extra checks:

class C:
    def __len__(self):
        return 'boo'

obj = C()
print(obj.__len__())  # fine
print(len(obj))       # error

or:

class C:
    def __str__(self):
        return 10

obj = C()
print(obj.__str__())  # fine
print(str(obj))       # error

As you see, when Python calls that magic methods implicitly, it does some extra checks as well.

---

4- This is the least important but using let's say len() on built-in data types such as str gives a little bit of speed as compared to __len__():

from timeit import timeit

string = 'abcdefghijklmn'

print(timeit("len(string)", globals=globals(), number=10_000_000))
print(timeit("string.__len__()", globals=globals(), number=10_000_000))

output:

0.5442426
0.8312854999999999

It's because of the lookup process(__len__ in the namespace), If you create a bound method before timing, it's gonna be faster.

bound_method = string.__len__
print(timeit("bound_method()", globals=globals(), number=10_000_000))

Python __index__ special method

Dict and List does not implement __getitem__ the same way. Dict objects uses a comparison (__eq__) on __hash__ of objects as key to use in __getitem__.

To make Thing usable for dict you have to implement both hash and eq.

Why does Python use 'magic methods'?

AFAIK, len is special in this respect and has historical roots.

Here's a quote from the FAQ:

Why does Python use methods for some
functionality (e.g. list.index()) but
functions for other (e.g. len(list))?

The major reason is history. Functions
were used for those operations that
were generic for a group of types and
which were intended to work even for
objects that didn’t have methods at
all (e.g. tuples). It is also
convenient to have a function that can
readily be applied to an amorphous
collection of objects when you use the
functional features of Python (map(),
apply() et al).

In fact, implementing len(), max(),
min() as a built-in function is
actually less code than implementing
them as methods for each type. One can
quibble about individual cases but
it’s a part of Python, and it’s too
late to make such fundamental changes
now. The functions have to remain to
avoid massive code breakage.

The other "magical methods" (actually called special method in the Python folklore) make lots of sense, and similar functionality exists in other languages. They're mostly used for code that gets called implicitly when special syntax is used.

For example:

• overloaded operators (exist in C++ and others)
• constructor/destructor
• hooks for accessing attributes
• tools for metaprogramming

and so on...

Where is the Python documentation for the special methods? (__init__, __new__, __len__, ...)

Please take a look at the special method names section in the Python language reference.

---

Related Topics