How to Decorate All Functions of a Class Without Typing It Over and Over for Each Method

How to decorate all functions of a class without typing it over and over for each method?

Decorate the class with a function that walks through the class's attributes and decorates callables. This may be the wrong thing to do if you have class variables that may happen to be callable, and will also decorate nested classes (credits to Sven Marnach for pointing this out) but generally it's a rather clean and simple solution. Example implementation (note that this will not exclude special methods (__init__ etc.), which may or may not be desired):

def for_all_methods(decorator):
    def decorate(cls):
        for attr in cls.__dict__: # there's propably a better way to do this
            if callable(getattr(cls, attr)):
                setattr(cls, attr, decorator(getattr(cls, attr)))
        return cls
    return decorate

Use like this:

@for_all_methods(mydecorator)
class C(object):
    def m1(self): pass
    def m2(self, x): pass
    ...

In Python 3.0 and 3.1, callable does not exist. It existed since forever in Python 2.x and is back in Python 3.2 as wrapper for isinstance(x, collections.Callable), so you can use that (or define your own callable replacement using this) in those versions.

Attaching a decorator to all functions within a class

The cleanest way to do this, or to do other modifications to a class definition, is to define a metaclass.

Alternatively, just apply your decorator at the end of the class definition using inspect:

import inspect

class Something:
    def foo(self): 
        pass

for name, fn in inspect.getmembers(Something, inspect.isfunction):
    setattr(Something, name, decorator(fn))

In practice of course you'll want to apply your decorator more selectively. As soon as you want to decorate all but one method you'll discover that it is easier and more flexible just to use the decorator syntax in the traditional way.

Apply decorator all function in a class without using Metaclass

Converting your metaclass to a class decorator should be straight forward. A class decorator simly receives the class as argument and returns the (modified) class:

def useless_class_decorator(cls):
    for name, method in cls.__dict__.items():
        if not name.startswith('_') and inspect.isroutine(method):
            setattr(cls, name, decorator(useless_decorator, method))
    return cls

The main difference here is that you can't direcly change cls.__dict__ here, as for new style classes that will be a dictproxy which does not support assignment, so you have to use setattr on the class instead. Then you simply create your class:

@useless_class_decorator
class Useless(object):
    def method_to_decorate(self, *args, *kwargs):
        ...

However this won't affect subclasses of Useless, those would also have to be decorated using the class decorator. If that's not acceptable, then a metaclass may be the better option...

Decorate all class methods without redecorating each time

I fixed it by changing to manipulating the instantiated object instead of the class itself.

My resource_wrappers if changed to for attr in dir(cls): and i pass cls to _function_wrapper using it as self.

For clarity, this is the final result of the example code:

UserResource.py:

class UserResource(object):
    def test(self):
        pass

resource.py:

def resource_wrapper(resource_class):
    def decorate(cls):
        for attr in dir(cls):
            if callable(getattr(cls, attr)) and not attr.startswith('_'):
                setattr(cls, attr, _function_wrapper(cls, getattr(cls, attr)))
        return cls
    return decorate(resource_class)

def _function_wrapper(cls, fn):
    from functools import wraps

    @wraps(fn)
    def wrapper(*args, **kwargs):
        self = cls
        arg = kwargs.pop('test', False)

        print arg  # See output further down

        return fn(*args, **kwargs)

    return wrapper

Usage:

r1 = resource_wrapper(UserResource())
r1.test(test=True)
# True

How to wrap every method of a class?

An elegant way to do it is described in Michael Foord's Voidspace blog in an entry about what metaclasses are and how to use them in the section titled A Method Decorating Metaclass. Simplifying it slightly and applying it to your situation resulted in this:

from functools import wraps
from types import FunctionType

def wrapper(method):
    @wraps(method)
    def wrapped(*args, **kwargs):
    #   ... <do something to/with "method" or the result of calling it>
    return wrapped

class MetaClass(type):
    def __new__(meta, classname, bases, classDict):
        newClassDict = {}
        for attributeName, attribute in classDict.items():
            if isinstance(attribute, FunctionType):
                # replace it with a wrapped version
                attribute = wrapper(attribute)
            newClassDict[attributeName] = attribute
        return type.__new__(meta, classname, bases, newClassDict)

class MyClass(object):
    __metaclass__ = MetaClass  # wrap all the methods
    def method1(self, ...):
        # ...etc ...

In Python, function/method decorators are just function wrappers plus some syntactic sugar to make using them easy (and prettier).

Python 3 Compatibility Update

The previous code uses Python 2.x metaclass syntax which would need to be translated in order to be used in Python 3.x, however it would then no longer work in the previous version. This means it would need to use:

class MyClass(metaclass=MetaClass)  # apply method-wrapping metaclass
    ...

instead of:

class MyClass(object):
    __metaclass__ = MetaClass  # wrap all the methods
    ...

If desired, it's possible to write code which is compatible with both Python 2.x and 3.x, but doing so requires using a slightly more complicated technique which dynamically creates a new base class that inherits the desired metaclass, thereby avoiding errors due to the syntax differences between the two versions of Python. This is basically what Benjamin Peterson's six module's with_metaclass() function does.

from types import FunctionType
from functools import wraps

def wrapper(method):
    @wraps(method)
    def wrapped(*args, **kwargs):
        print('{!r} executing'.format(method.__name__))
        return method(*args, **kwargs)
    return wrapped

class MetaClass(type):
    def __new__(meta, classname, bases, classDict):
        newClassDict = {}
        for attributeName, attribute in classDict.items():
            if isinstance(attribute, FunctionType):
                # replace it with a wrapped version
                attribute = wrapper(attribute)
            newClassDict[attributeName] = attribute
        return type.__new__(meta, classname, bases, newClassDict)

def with_metaclass(meta):
    """ Create an empty class with the supplied bases and metaclass. """
    return type.__new__(meta, "TempBaseClass", (object,), {})

if __name__ == '__main__':

    # Inherit metaclass from a dynamically-created base class.
    class MyClass(with_metaclass(MetaClass)):
        @staticmethod
        def a_static_method():
            pass

        @classmethod
        def a_class_method(cls):
            pass

        def a_method(self):
            pass

    instance = MyClass()
    instance.a_static_method()  # Not decorated.
    instance.a_class_method()   # Not decorated.
    instance.a_method()         # -> 'a_method' executing

How to set up a class with all the methods of and functions like a built in such as float, but holds onto extra data?

I suggest subclassing float and using a couple decorators to "capture" the float output from any method (except for __new__ of course) and returning a Datum object instead of a float object.

First we write the method decorator (which really isn't being used as a decorator below, it's just a function that modifies the output of another function, AKA a wrapper function):

def mydecorator(f,cls):
    #f is the method being modified, cls is its class (in this case, Datum)
    def func_wrapper(*args,**kwargs):
        #*args and **kwargs are all the arguments that were passed to f
        newvalue = f(*args,**kwargs)
        #newvalue now contains the output float would normally produce
        ##Now get cls instance provided as part of args (we need one
        ##if we're going to reattach instance information later):
        try:
            self = args[0]
            ##Now check to make sure new value is an instance of some numerical 
            ##type, but NOT a bool or a cls type (which might lead to recursion)
            ##Including ints so things like modulo and round will work right
            if (isinstance(newvalue,float) or isinstance(newvalue,int)) and not isinstance(newvalue,bool) and type(newvalue) != cls:
                ##If newvalue is a float or int, now we make a new cls instance using the
                ##newvalue for value and using the previous self instance information (arg[0])
                ##for the other fields
                return cls(newvalue,self.dtype,self.source,self.index1,self.index2)
        #IndexError raised if no args provided, AttributeError raised of self isn't a cls instance
        except (IndexError, AttributeError): 
            pass
        ##If newvalue isn't numerical, or we don't have a self, just return what
        ##float would normally return
        return newvalue
    #the function has now been modified and we return the modified version
    #to be used instead of the original version, f
    return func_wrapper

The first decorator only applies to a method to which it is attached. But we want it to decorate all (actually, almost all) the methods inherited from float (well, those that appear in the float's __dict__, anyway). This second decorator will apply our first decorator to all of the methods in the float subclass except for those listed as exceptions (see this answer):

def for_all_methods_in_float(decorator,*exceptions):
    def decorate(cls):
        for attr in float.__dict__:
            if callable(getattr(float, attr)) and not attr in exceptions:
                setattr(cls, attr, decorator(getattr(float, attr),cls))
        return cls
    return decorate

Now we write the subclass much the same as you had before, but decorated, and excluding __new__ from decoration (I guess we could also exclude __init__ but __init__ doesn't return anything, anyway):

@for_all_methods_in_float(mydecorator,'__new__')
class Datum(float):
    def __new__(klass, value, dtype="dtype", source="source", index1="index1", index2="index2"):
        return super(Datum,klass).__new__(klass,value)
    def __init__(self, value, dtype="dtype", source="source", index1="index1", index2="index2"):
        self.value = value
        self.dtype = dtype
        self.source = source
        self.index1 = index1
        self.index2 = index2
        super(Datum,self).__init__()

Here are our testing procedures; iteration seems to work correctly:

d1 = Datum(1.5)
d2 = Datum(3.2)
d3 = d1+d2
assert d3.source == 'source'
L=[d1,d2,d3]
d4=max(L)
assert d4.source == 'source'
L = [i for i in L]
assert L[0].source == 'source'
assert type(L[0]) == Datum
minimum = min(L)
assert [x - minimum for x in L][0].source == 'source'

Notes:

• I am using Python 3. Not certain if that will make a difference for you.
• This approach effectively overrides EVERY method of float other than the exceptions, even the ones for which the result isn't modified. There may be side effects to this (subclassing a built-in and then overriding all of its methods), e.g. a performance hit or something; I really don't know.
• This will also decorate nested classes.
• This same approach could also be implemented using a metaclass.

How to add methods of class to a list inside the class with a decorator

Your current approach fails because when transform_meth is called, TextProcessor isn't bound to anything yet (or if it is, that object gets overwritten when the class statement completes).

The simple solution would be to define transform_meth inside the class statement, so that it could simply declare transforms as a nonlocal variable. However, that won't work because a class statement doesn't establish a new scope.

Instead, you can define a function that creates the decorator, which takes the desired list (at that point a just a name in the body of the class statement, not from any assumed scope). That function returns a closure over the list argument
so that you can append to it.

def make_decorator(lst):

    # *This* will be the function bound to the name 'transform_meth'
    def _(meth):
        lst.append(meth)
        return meth

    return _

class TextProcessor:
    transforms: List[callable] = []

    transform_meth = make_decorator(transforms)

    @transform_meth
    def m1(self, text):
        return text

    @transform_meth
    def m2(self, text):
        return text

    def transform(self, text):
        for transform in self.transforms:
            text = transform(text)
        return text

    del transform_meth  # Not needed anymore, don't create a class attribute

How to get all methods of a Python class with given decorator?

Method 1: Basic registering decorator

I already answered this question here: Calling functions by array index in Python =)

---

Method 2: Sourcecode parsing

If you do not have control over the class definition, which is one interpretation of what you'd like to suppose, this is impossible (without code-reading-reflection), since for example the decorator could be a no-op decorator (like in my linked example) that merely returns the function unmodified. (Nevertheless if you allow yourself to wrap/redefine the decorators, see Method 3: Converting decorators to be "self-aware", then you will find an elegant solution)

It is a terrible terrible hack, but you could use the inspect module to read the sourcecode itself, and parse it. This will not work in an interactive interpreter, because the inspect module will refuse to give sourcecode in interactive mode. However, below is a proof of concept.

#!/usr/bin/python3

import inspect

def deco(func):
    return func

def deco2():
    def wrapper(func):
        pass
    return wrapper

class Test(object):
    @deco
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

def methodsWithDecorator(cls, decoratorName):
    sourcelines = inspect.getsourcelines(cls)[0]
    for i,line in enumerate(sourcelines):
        line = line.strip()
        if line.split('(')[0].strip() == '@'+decoratorName: # leaving a bit out
            nextLine = sourcelines[i+1]
            name = nextLine.split('def')[1].split('(')[0].strip()
            yield(name)

It works!:

>>> print(list(  methodsWithDecorator(Test, 'deco')  ))
['method']

Note that one has to pay attention to parsing and the python syntax, e.g. @deco and @deco(... are valid results, but @deco2 should not be returned if we merely ask for 'deco'. We notice that according to the official python syntax at http://docs.python.org/reference/compound_stmts.html decorators are as follows:

decorator      ::=  "@" dotted_name ["(" [argument_list [","]] ")"] NEWLINE

We breathe a sigh of relief at not having to deal with cases like @(deco). But note that this still doesn't really help you if you have really really complicated decorators, such as @getDecorator(...), e.g.

def getDecorator():
    return deco

Thus, this best-that-you-can-do strategy of parsing code cannot detect cases like this. Though if you are using this method, what you're really after is what is written on top of the method in the definition, which in this case is getDecorator.

According to the spec, it is also valid to have @foo1.bar2.baz3(...) as a decorator. You can extend this method to work with that. You might also be able to extend this method to return a <function object ...> rather than the function's name, with lots of effort. This method however is hackish and terrible.

---

Method 3: Converting decorators to be "self-aware"

If you do not have control over the decorator definition (which is another interpretation of what you'd like), then all these issues go away because you have control over how the decorator is applied. Thus, you can modify the decorator by wrapping it, to create your own decorator, and use that to decorate your functions. Let me say that yet again: you can make a decorator that decorates the decorator you have no control over, "enlightening" it, which in our case makes it do what it was doing before but also append a .decorator metadata property to the callable it returns, allowing you to keep track of "was this function decorated or not? let's check function.decorator!". And then you can iterate over the methods of the class, and just check to see if the decorator has the appropriate .decorator property! =) As demonstrated here:

def makeRegisteringDecorator(foreignDecorator):
    """
        Returns a copy of foreignDecorator, which is identical in every
        way(*), except also appends a .decorator property to the callable it
        spits out.
    """
    def newDecorator(func):
        # Call to newDecorator(method)
        # Exactly like old decorator, but output keeps track of what decorated it
        R = foreignDecorator(func) # apply foreignDecorator, like call to foreignDecorator(method) would have done
        R.decorator = newDecorator # keep track of decorator
        #R.original = func         # might as well keep track of everything!
        return R

    newDecorator.__name__ = foreignDecorator.__name__
    newDecorator.__doc__ = foreignDecorator.__doc__
    # (*)We can be somewhat "hygienic", but newDecorator still isn't signature-preserving, i.e. you will not be able to get a runtime list of parameters. For that, you need hackish libraries...but in this case, the only argument is func, so it's not a big issue

    return newDecorator

Demonstration for @decorator:

deco = makeRegisteringDecorator(deco)

class Test2(object):
    @deco
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

def methodsWithDecorator(cls, decorator):
    """ 
        Returns all methods in CLS with DECORATOR as the
        outermost decorator.

        DECORATOR must be a "registering decorator"; one
        can make any decorator "registering" via the
        makeRegisteringDecorator function.
    """
    for maybeDecorated in cls.__dict__.values():
        if hasattr(maybeDecorated, 'decorator'):
            if maybeDecorated.decorator == decorator:
                print(maybeDecorated)
                yield maybeDecorated

It works!:

>>> print(list(   methodsWithDecorator(Test2, deco)   ))
[<function method at 0x7d62f8>]

However, a "registered decorator" must be the outermost decorator, otherwise the .decorator attribute annotation will be lost. For example in a train of

@decoOutermost
@deco
@decoInnermost
def func(): ...

you can only see metadata that decoOutermost exposes, unless we keep references to "more-inner" wrappers.

sidenote: the above method can also build up a .decorator that keeps track of the entire stack of applied decorators and input functions and decorator-factory arguments. =) For example if you consider the commented-out line R.original = func, it is feasible to use a method like this to keep track of all wrapper layers. This is personally what I'd do if I wrote a decorator library, because it allows for deep introspection.

There is also a difference between @foo and @bar(...). While they are both "decorator expressons" as defined in the spec, note that foo is a decorator, while bar(...) returns a dynamically-created decorator, which is then applied. Thus you'd need a separate function makeRegisteringDecoratorFactory, that is somewhat like makeRegisteringDecorator but even MORE META:

def makeRegisteringDecoratorFactory(foreignDecoratorFactory):
    def newDecoratorFactory(*args, **kw):
        oldGeneratedDecorator = foreignDecoratorFactory(*args, **kw)
        def newGeneratedDecorator(func):
            modifiedFunc = oldGeneratedDecorator(func)
            modifiedFunc.decorator = newDecoratorFactory # keep track of decorator
            return modifiedFunc
        return newGeneratedDecorator
    newDecoratorFactory.__name__ = foreignDecoratorFactory.__name__
    newDecoratorFactory.__doc__ = foreignDecoratorFactory.__doc__
    return newDecoratorFactory

Demonstration for @decorator(...):

def deco2():
    def simpleDeco(func):
        return func
    return simpleDeco

deco2 = makeRegisteringDecoratorFactory(deco2)

print(deco2.__name__)
# RESULT: 'deco2'

@deco2()
def f():
    pass

This generator-factory wrapper also works:

>>> print(f.decorator)
<function deco2 at 0x6a6408>

bonus Let's even try the following with Method #3:

def getDecorator(): # let's do some dispatching!
    return deco

class Test3(object):
    @getDecorator()
    def method(self):
        pass

    @deco2()
    def method2(self):
        pass

Result:

>>> print(list(   methodsWithDecorator(Test3, deco)   ))
[<function method at 0x7d62f8>]

As you can see, unlike method2, @deco is correctly recognized even though it was never explicitly written in the class. Unlike method2, this will also work if the method is added at runtime (manually, via a metaclass, etc.) or inherited.

Be aware that you can also decorate a class, so if you "enlighten" a decorator that is used to both decorate methods and classes, and then write a class within the body of the class you want to analyze, then methodsWithDecorator will return decorated classes as well as decorated methods. One could consider this a feature, but you can easily write logic to ignore those by examining the argument to the decorator, i.e. .original, to achieve the desired semantics.

Can a decorator of an instance method access the class?

If you are using Python 2.6 or later you could use a class decorator, perhaps something like this (warning: untested code).

def class_decorator(cls):
   for name, method in cls.__dict__.iteritems():
        if hasattr(method, "use_class"):
            # do something with the method and class
            print name, cls
   return cls

def method_decorator(view):
    # mark the method as something that requires view's class
    view.use_class = True
    return view

@class_decorator
class ModelA(object):
    @method_decorator
    def a_method(self):
        # do some stuff
        pass

The method decorator marks the method as one that is of interest by adding a "use_class" attribute - functions and methods are also objects, so you can attach additional metadata to them.

After the class has been created the class decorator then goes through all the methods and does whatever is needed on the methods that have been marked.

If you want all the methods to be affected then you could leave out the method decorator and just use the class decorator.

How can I decorate an instance method with a decorator class?

tl;dr

You can fix this problem by making the Timed class a descriptor and returning a partially applied function from __get__ which applies the Test object as one of the arguments, like this

class Timed(object):
    def __init__(self, f):
        self.func = f

    def __call__(self, *args, **kwargs):
        print(self)
        start = dt.datetime.now()
        ret = self.func(*args, **kwargs)
        time = dt.datetime.now() - start
        ret["time"] = time
        return ret

    def __get__(self, instance, owner):
        from functools import partial
        return partial(self.__call__, instance)

---

The actual problem

Quoting Python documentation for decorator,

The decorator syntax is merely syntactic sugar, the following two function definitions are semantically equivalent:

def f(...):
    ...
f = staticmethod(f)

@staticmethod
def f(...):
    ...

So, when you say,

@Timed
def decorated(self, *args, **kwargs):

it is actually

decorated = Timed(decorated)

only the function object is passed to the Timed, the object to which it is actually bound is not passed on along with it. So, when you invoke it like this

ret = self.func(*args, **kwargs)

self.func will refer to the unbound function object and it is invoked with Hello as the first argument. That is why self prints as Hello.

---

How can I fix this?

Since you have no reference to the Test instance in the Timed, the only way to do this would be to convert Timed as a descriptor class. Quoting the documentation, Invoking descriptors section,

In general, a descriptor is an object attribute with “binding behavior”, one whose attribute access has been overridden by methods in the descriptor protocol: __get__(), __set__(), and __delete__(). If any of those methods are defined for an object, it is said to be a descriptor.

The default behavior for attribute access is to get, set, or delete the attribute from an object’s dictionary. For instance, a.x has a lookup chain starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing through the base classes of type(a) excluding metaclasses.

However, if the looked-up value is an object defining one of the descriptor methods, then Python may override the default behavior and invoke the descriptor method instead.

We can make Timed a descriptor, by simply defining a method like this

def __get__(self, instance, owner):
    ...

Here, self refers to the Timed object itself, instance refers to the actual object on which the attribute lookup is happening and owner refers to the class corresponding to the instance.

Now, when __call__ is invoked on Timed, the __get__ method will be invoked. Now, somehow, we need to pass the first argument as the instance of Test class (even before Hello). So, we create another partially applied function, whose first parameter will be the Test instance, like this

def __get__(self, instance, owner):
    from functools import partial
    return partial(self.__call__, instance)

Now, self.__call__ is a bound method (bound to Timed instance) and the second parameter to partial is the first argument to the self.__call__ call.

So, all these effectively translate like this

t.call_deco()
self.decorated("Hello", world="World")

Now self.decorated is actually Timed(decorated) (this will be referred as TimedObject from now on) object. Whenever we access it, the __get__ method defined in it will be invoked and it returns a partial function. You can confirm that like this

def call_deco(self):
    print(self.decorated)
    self.decorated("Hello", world="World")

would print

<functools.partial object at 0x7fecbc59ad60>
...

So,

self.decorated("Hello", world="World")

gets translated to

Timed.__get__(TimedObject, <Test obj>, Test.__class__)("Hello", world="World")

Since we return a partial function,

partial(TimedObject.__call__, <Test obj>)("Hello", world="World"))

which is actually

TimedObject.__call__(<Test obj>, 'Hello', world="World")

So, <Test obj> also becomes a part of *args, and when self.func is invoked, the first argument will be the <Test obj>.

---

Related Topics