Python: Changing Methods and Attributes at Runtime

Python: changing methods and attributes at runtime

I wish to create a class in Python that I can add and remove attributes and methods.

import types

class SpecialClass(object):
    @classmethod
    def removeVariable(cls, name):
        return delattr(cls, name)

    @classmethod
    def addMethod(cls, func):
        return setattr(cls, func.__name__, types.MethodType(func, cls))

def hello(self, n):
    print n

instance = SpecialClass()
SpecialClass.addMethod(hello)

>>> SpecialClass.hello(5)
5

>>> instance.hello(6)
6

>>> SpecialClass.removeVariable("hello")

>>> instance.hello(7)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'SpecialClass' object has no attribute 'hello'

>>> SpecialClass.hello(8)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'SpecialClass' has no attribute 'hello'

Change Python Class attribute dynamically

class attributes can be read on the class or an instance, but you can only set them on the class (trying to set them on an instance will only create an instance attribute that will shadow the class attribute).

If the condition is known at import time, you can just test it in the class body:

xxx = True 

class A(object):
   cls_attr = 'value'

class B(A):
   if xxx:
       cls_attr = 'this_value'
   else
       cls_attr = 'that_value'

Now if you want to change it during the program's execution, you either have to use a classmethod:

class B(A):
   @classmethod
   def set_cls_attr(cls, xxx):   
       if xxx:
           cls.cls_attr = 'this_value'
       else:
           cls.cls_attr = 'that_value'

or if you need to access your instance during the test:

class B(A):
   def set_cls_attr(self, xxx):   
       cls = type(self)
       if xxx:
           cls.cls_attr = 'this_value'
       else:
           cls.cls_attr = 'that_value'

How to change class instance behaviour at runtime in-place?

Something like that, I think:

class Cat:
    def __init__(self, foo):
        self.foo = foo

    def do_things(self):
        print('Do cat things')

    def turn_to_zombie(self, bar):
        self.bar = bar
        self.__class__ = ZombieCat

class ZombieCat(Cat):
    def __init__(self, foo, bar):
        super().__init__(foo)
        self.bar = bar

    def do_things(self):
        print('Do zombie cat things')

class SchroedingerCat:
    _cat = Cat
    _zombie = ZombieCat
    _schroedinger = None

    def __init__(self, foo, bar=None):
        if bar is not None:
            self._schroedinger = self._zombie(foo, bar)
        else:
            self._schroedinger = self._cat(foo)

    def turn_to_zombie(self, bar):
        self._schroedinger = self._zombie(self._schroedinger.foo, bar)
        return self

    def __getattr__(self, name):
        return getattr(self._schroedinger, name)

SchroedingerCat('aaa').do_things()
SchroedingerCat('aaa').turn_to_zombie('bbb').do_things()

Self merged classes is too complex and not intuitive, I think.

Dark magic attention Do not use it, but it's worked:

from functools import partial

class DarkMagicCat:
    _cat = Cat
    _zombie = ZombieCat
    _schroedinger = None

    def __init__(self, foo, bar):
        self.foo = foo
        self.bar = bar
        self._schroedinger = self._cat

    def turn_to_zombie(self, bar):
        self._schroedinger = self._zombie
        return self

    def __getattr__(self, name):
        return partial(getattr(self._schroedinger, name), self=self)

Change python mro at runtime

The other provided answers are advisable if you are not bound by the constraints mentioned in the question. Otherwise, we need to take a journey into mro hacks and metaclass land.

After some reading, I discovered you can change the mro of a class, using a metaclass.

This however, is at class creation time, not at object creation time. Slight modification is necessary.

The metaclass provides the mro method, which we overload, that is called during class creation (the metaclass' __new__ call) to produce the __mro__ attribute.

The __mro__ attribute is not a normal attribute, in that:

1. It is read only
2. It is defined BEFORE the metaclass' __new__ call

However, it appears to be recalculated (using the mro method) when a class' base is changed. This forms the basis of the hack.

In brief:

• The subclass (B) is created using a metaclass (change_mro_meta). This metaclass provides:
  
  
  • An overloaded mro method
  • Class methods to change the __mro__ attribute
  • A class attribute (change_mro) to control the mro behaviour

As mentioned, modifying the mro of a class while in its __init__ is not thread safe.

The following may disturb some viewers. Viewer discretion is advised.

The hack:

class change_mro_meta(type):
    def __new__(cls, cls_name, cls_bases, cls_dict):
        out_cls = super(change_mro_meta, cls).__new__(cls, cls_name, cls_bases, cls_dict)
        out_cls.change_mro = False
        out_cls.hack_mro   = classmethod(cls.hack_mro)
        out_cls.fix_mro    = classmethod(cls.fix_mro)
        out_cls.recalc_mro = classmethod(cls.recalc_mro)
        return out_cls

    @staticmethod
    def hack_mro(cls):
        cls.change_mro = True
        cls.recalc_mro()

    @staticmethod
    def fix_mro(cls):
        cls.change_mro = False
        cls.recalc_mro()

    @staticmethod
    def recalc_mro(cls):
        # Changing a class' base causes __mro__ recalculation
        cls.__bases__  = cls.__bases__ + tuple()

    def mro(cls):
        default_mro = super(change_mro_meta, cls).mro()
        if hasattr(cls, "change_mro") and cls.change_mro:
            return default_mro[1:2] + default_mro
        else:
            return default_mro

class A(object):
    def __init__(self):
        print "__init__ A"
        self.hello()

    def hello(self):
        print "A hello"

class B(A):
    __metaclass__ = change_mro_meta
    def __init__(self):
        self.hack_mro()
        super(B, self).__init__()
        self.fix_mro()
        print "__init__ B"
        self.msg_str = "B"
        self.hello()

    def hello(self):
        print "%s hello" % self.msg_str

a = A()
b = B()

Some notes:

The hack_mro, fix_mro and recalc_mro methods are staticmethods to the metaclass but classmethods to the class. It did this, instead of multiple inheritance, because I wanted to group the mro code together.

The mro method itself returns the default ordinarily. Under the hack condition, it appends the second element of the default mro (the immediate parent class) to the mro, thereby causing the parent class to see its own methods first before the subclass'.

I'm unsure of the portability of this hack. Its been tested on 64bit CPython 2.7.3 running on Windows 7 64bit.

Don't worry, I'm sure this won't end up in production code somewhere.

Adding a method to an existing object instance

In Python, there is a difference between functions and bound methods.

>>> def foo():
...     print "foo"
...
>>> class A:
...     def bar( self ):
...         print "bar"
...
>>> a = A()
>>> foo
<function foo at 0x00A98D70>
>>> a.bar
<bound method A.bar of <__main__.A instance at 0x00A9BC88>>
>>>

Bound methods have been "bound" (how descriptive) to an instance, and that instance will be passed as the first argument whenever the method is called.

Callables that are attributes of a class (as opposed to an instance) are still unbound, though, so you can modify the class definition whenever you want:

>>> def fooFighters( self ):
...     print "fooFighters"
...
>>> A.fooFighters = fooFighters
>>> a2 = A()
>>> a2.fooFighters
<bound method A.fooFighters of <__main__.A instance at 0x00A9BEB8>>
>>> a2.fooFighters()
fooFighters

Previously defined instances are updated as well (as long as they haven't overridden the attribute themselves):

>>> a.fooFighters()
fooFighters

The problem comes when you want to attach a method to a single instance:

>>> def barFighters( self ):
...     print "barFighters"
...
>>> a.barFighters = barFighters
>>> a.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: barFighters() takes exactly 1 argument (0 given)

The function is not automatically bound when it's attached directly to an instance:

>>> a.barFighters
<function barFighters at 0x00A98EF0>

To bind it, we can use the MethodType function in the types module:

>>> import types
>>> a.barFighters = types.MethodType( barFighters, a )
>>> a.barFighters
<bound method ?.barFighters of <__main__.A instance at 0x00A9BC88>>
>>> a.barFighters()
barFighters

This time other instances of the class have not been affected:

>>> a2.barFighters()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: A instance has no attribute 'barFighters'

More information can be found by reading about descriptors and metaclass programming.

In Python can I change the structure of a class at runtime?

You could set the attribute of the class instead of the instance.

setattr(Foo, "baz", "spam")

Output

>>> class Foo:
...     def __init__(self, bar):
...         self.bar = bar

>>> f = Foo('bar')
>>> setattr(Foo, "baz", "spam")
>>> f.baz
'spam'
>>> f2 = Foo('bar')
>>> f2.baz
'spam'

How to dynamically change base class of instances at runtime?

Ok, again, this is not something you should normally do, this is for informational purposes only.

Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:

setattr(Person, '__mro__', (Person, Friendly, object))

The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.

A possible workaround is to simply redefine the class:

def modify_Person_to_be_friendly():
    # so that we're modifying the global identifier 'Person'
    global Person

    # now just redefine the class using type(), specifying that the new
    # class should inherit from Friendly and have all attributes from
    # our old Person class
    Person = type('Person', (Friendly,), dict(Person.__dict__)) 

def main():
    modify_Person_to_be_friendly()
    p = Person()
    p.hello()  # works!

What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):

def main():
    oldperson = Person()
    ModifyPersonToBeFriendly()
    p = Person()
    p.hello()  
    # works!  But:
    oldperson.hello()
    # does not

If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.

Dynamically update attributes of an object that depend on the state of other attributes of same object

Yes, there is! It's called properties.

Read Only Properties

class Test(object):
    def __init__(self,a,b):
        self.a = a
        self.b = b
    @property
    def c(self):
        return self.a + self.b

With the above code, c is now a read-only property of the Test class.

Mutable Properties

You can also give a property a setter, which would make it read/write and allow you to set its value directly. It would look like this:

class Test(object):
    def __init__(self, c = SomeDefaultValue):
        self._c = SomeDefaultValue
    @property
    def c(self):
        return self._c
    @c.setter
    def c(self,value):
        self._c = value

However, in this case, it would not make sense to have a setter for self.c, since its value depends on self.a and self.b.

What does @property mean?

The @property bit is an example of something called a decorator. A decorator actually wraps the function (or class) it decorates into another function (the decorator function). After a function has been decorated, when it is called it is actually the decorator that is called with the function (and its arguments) as an argument. Usually (but not always!) the decorated function does something interesting, and then calls the original (decorated) function like it would normally. For example:

def my_decorator(thedecoratedfunction):
    def wrapped(*allofthearguments):
        print("This function has been decorated!") #something interesting
        thedecoratedfunction(*allofthearguments)   #calls the function as normal
    return wrapped

@my_decorator
def myfunction(arg1, arg2):
    pass

This is equivalent to:

def myfunction(arg1, arg2):
    pass
myfunction = my_decorator(myfunction)

So this means in the class example above, instead of using the decorator you could also do this:

def c(self):
    return self.a + self.b
c = property(c)

They are exactly the same thing. The @property is just syntactic sugar to replace calls for myobject.c with the property getter and setter (deleters are also an option).

Wait - How does that work?

You might be wondering why simply doing this once:

myfunction = my_decorator(myfunction)

...results in such a drastic change! So that, from now on, when calling:

myfunction(arg1, arg2)

...you are actually calling my_decorator(myfunction), with arg1, arg2 sent to the interior wrapped function inside of my_decorator. And not only that, but all of this happens even though you didn't even mention my_decorator or wrapped in your function call at all!

All of this works by virtue of something called a closure. When the function is passed into the decorator in this way (e.g., property(c)), the function's name is re-bound to the wrapped version of the function instead of the original function, and the original function's arguments are always passed to wrapped instead of the original function. This is simply the way that closures work, and there's nothing magical about it. Here is some more information about closures.

Descriptors

So to summarize so far: @property is just a way of wrapping the class method inside of the property() function so the wrapped class method is called instead of the original, unwrapped class method. But what is the property function? What does it do?

The property function adds something called a descriptor to the class. Put simply, a descriptor is an object class that can have separate get, set, and delete methods. When you do this:

@property
def c(self):
    return self._c

...you are adding a descriptor to the Test class called c, and defining the get method (actually, __get__()) of the c descriptor as equal to the c(self) method.

When you do this:

@c.setter
def c(self,value):
    self._c

...you are defining the set method (actually, __set__()) of the c descriptor as equal to the c(self,value) method.

Summary

An amazing amount of stuff is accomplished by simply adding @property to your def c(self) method! In practice, you probably don't need to understand all of this right away to begin using it. However, I recommend keeping in mind that when you use @property, you are using decorators, closures, and descriptors, and if you are at all serious about learning Python it would be well worth your time to investigate each of these topics on their own.

How to change one class in an external package for my use?

In Python, a method is just an attribute of the class object that happens to be a function having self as its first parameter. That means that you can easily replace it by your own function, provided you keep the same signature. You can even call the original method from your own one:

# after having imported InnerThing
_orig_inner_method1 = InnerThing.inner_method1

def _patched_inner_method1(self, ...):
    # your own code
    ...
    _orig_inner_method1(self, ...)   # eventually call the original method
    ...

InnerThing.inner_method1 = _patched_inner_method1

From that point, any InnerThing object created will use your patched method.

---

Related Topics