Extension Method for Python Built-In Types

Extension method for python built-in types

It can be done in pure Python with this incredibly clever module:

https://pypi.python.org/pypi/forbiddenfruit

For example:

import functools
import ctypes
import __builtin__
import operator

class PyObject(ctypes.Structure):
    pass

Py_ssize_t = hasattr(ctypes.pythonapi, 'Py_InitModule4_64') and ctypes.c_int64 or ctypes.c_int

PyObject._fields_ = [
    ('ob_refcnt', Py_ssize_t),
    ('ob_type', ctypes.POINTER(PyObject)),
]

class SlotsPointer(PyObject):
    _fields_ = [('dict', ctypes.POINTER(PyObject))]

def proxy_builtin(klass):
    name = klass.__name__
    slots = getattr(klass, '__dict__', name)

    pointer = SlotsPointer.from_address(id(slots))
    namespace = {}

    ctypes.pythonapi.PyDict_SetItem(
        ctypes.py_object(namespace),
        ctypes.py_object(name),
        pointer.dict,
    )

    return namespace[name]

def die(message, cls=Exception):
    """
        Raise an exception, allows you to use logical shortcut operators to test for object existence succinctly.

        User.by_name('username') or die('Failed to find user')
    """
    raise cls(message)

def unguido(self, key):
    """
        Attempt to find methods which should really exist on the object instance.
    """
    return functools.partial((getattr(__builtin__, key, None) if hasattr(__builtin__, key) else getattr(operator, key, None)) or die(key, KeyError), self)

class mapper(object):
    def __init__(self, iterator, key):
        self.iterator = iterator
        self.key = key
        self.fn = lambda o: getattr(o, key)

    def __getattribute__(self, key):
        if key in ('iterator', 'fn', 'key'): return object.__getattribute__(self, key)
        return mapper(self, key)

    def __call__(self, *args, **kwargs):
        self.fn = lambda o: (getattr(o, self.key, None) or unguido(o, self.key))(*args, **kwargs)
        return self

    def __iter__(self):
        for value in self.iterator:
            yield self.fn(value)

class foreach(object):
    """
        Creates an output iterator which will apply any functions called on it to every element
        in the input iterator. A kind of chainable version of filter().

        E.g:

        foreach([1, 2, 3]).__add__(2).__str__().replace('3', 'a').upper()

        is equivalent to:

        (str(o + 2).replace('3', 'a').upper() for o in iterator)

        Obviously this is not 'Pythonic'.
    """
    def __init__(self, iterator):
        self.iterator = iterator

    def __getattribute__(self, key):
        if key in ('iterator',): return object.__getattribute__(self, key)
        return mapper(self.iterator, key)

    def __iter__(self):
        for value in self.iterator:
            yield value

proxy_builtin(list)['foreach'] = property(foreach)

import string

print string.join([1, 2, 3].foreach.add(2).str().add(' cookies').upper(), ', ')

>>> 3 COOKIES, 4 COOKIES, 5 COOKIES

There, doesn't that feel good?

Extension methods in Python

You can add whatever methods you like on class objects defined in Python code (AKA monkey patching):

>>> class A(object):
>>>     pass

>>> def stuff(self):
>>>     print self

>>> A.test = stuff
>>> A().test()

This does not work on builtin types, because their __dict__ is not writable (it's a dictproxy).

So no, there is no "real" extension method mechanism in Python.

Extending builtin classes in python

Just subclass the type

>>> class X(str):
...     def my_method(self):
...         return int(self)
...
>>> s = X("Hi Mom")
>>> s.lower()
'hi mom'
>>> s.my_method()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 3, in my_method
ValueError: invalid literal for int() with base 10: 'Hi Mom'

>>> z = X("271828")
>>> z.lower()
'271828'
>>> z.my_method()
271828

Class extension in python like in swift?

For the sake of completeness, I think this is what you should do (In Swift as well!):

class Meter(float):
    def to_km(self): return self * 1000.0 
    def to_m (self): return self 
    def to_cm(self): return self / 100.0 
    def to_mm(self): return self / 1000.0 
    def to_ft(self): return self / 3.28084 
oneInch = Meter(25.4).to_mm()
print(oneInch)

Make it clear that your object represents a meter, and that you are converting it to something.

If you want some syntactic sugar, that I am not sure is helpful, you can override the item getter so you do not have to use ():

class Meter(float):
    conversions = {
        'km':1000,
        'cm':.01,
        'mm':.001,
        'ft':1/3.28084
    }
    def __getattr__(self,x):
        try:
            return self*Meter.conversions[x]
        except KeyError:
            raise KeyError("No such conversion!")
oneInch = Meter(25.4).mm
print(oneInch)

Adding conversions is as simple as:

Meter.conversions['whatever'] = 123.123

How do I create new methods and attributes for built-in datatypes in python?

This can't be done, since str is one of the immutable types in Python. Among the immutable types are bool, int, float, tuple, string, frozenset and perhaps a few more I don't know about.

With other types, you might be able to do something like this:

class MyClass:
    def __str__(self):
        return "my class"

def print_me_quoted(self):
    print(f'"{self}"')

mc_old = MyClass()
MyClass.print_me_quoted = print_me_quoted
mc_new = MyClass()

# both now have it
mc_old.print_me_quoted()
mc_new.print_me_quoted()

It's still a spectacularly bad idea, to change the behaviour of a class by monkey-patching it like this, but it can be done. Expect your editor or IDE not to like it, or to understand it - you'll see warnings.

If you try this with str:

def print_me_quoted(self):
    print(f'"{self}"')

str.print_me_quoted = print_me_quoted
mc = str(10)
mc.print_me_quoted()

You get this TypeError:

TypeError: cannot set 'print_me_quoted' attribute of immutable type 'str'

You could do it on your own version of str:

class Str(str):
    ...

def print_me_quoted(self):
    print(f'"{self}"')

Str.print_me_quoted = print_me_quoted
mc = Str(10)
mc.print_me_quoted()

But of course that does not change the standard behaviour of strings - which is exactly the point.

Override built-in types in Python

You can monkey-patch a lot, but not anything that's implemented in C, which includes the built-in str class.

>>> str.__add__ = lambda x, y: 'xyzzy'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type 'str'

Similarly, you can't override __add__ on collections.defaultdict:

>>> from collections import defaultdict
>>> defaultdict.__add__ = lambda x, y: 'xyzzy'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type 'collections.defaultdict'

Once you get away from the very core, though, you can often override things, even in the standard library:

>>> from random import Random
>>> Random.__add__ = lambda x, y: 'xyzzy'
>>> Random() + Random()
'xyzzy'

You still can't override things that are implemented in C, even if they're third-party libraries:

>>> import numpy as np
>>> np.ndarray.__add__ = lambda x, y: 'xyzzy'
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: can't set attributes of built-in/extension type 'numpy.ndarray'

---

Related Topics