Python Decorator Handling Docstrings

Python decorator handling docstrings

Use functools.wraps() to update the attributes of the decorator:

from functools import wraps

def decorator(f):
    @wraps(f)
    def _decorator():
        print 'decorator active'
        f()
    return _decorator

@decorator
def foo():
    '''the magic foo function'''
    print 'this is function foo'

help(foo)

Also see the Standard Library documentation for functools.

Where to put the doc string for a decorator

Put the documentation for the actually decorator in the top-level decorator function. When a user is attempting to use your decorator, this is where they would expect to find the documentation for it. For example, take the functools.singledispatch decorator from the standard library:

>>> from functools import singledispatch
>>>
>>> print(singledispatch.__doc__) # the decorator has it's documentation in the actually decorator function
Single-dispatch generic function decorator.

    Transforms a function into a generic function, which can have different
    behaviours depending upon the type of its first argument. The decorated
    function acts as the default implementation, and additional
    implementations can be registered using the register() attribute of the
    generic function.

>>>

However, you should also use functools.wraps to transfer any documentation contained in the decorated function to the wrapper function:

>>> from functools import wraps
>>>
>>> def dec(func):
...     @wraps(func)
...     def wrap(*args, **kwargs):
...         return func(*args, **kwargs)
...     return wrap
...
>>> @dec
... def func():
...     """Docstring here"""
...
>>> print(func.__doc__)
Docstring here
>>>

How to make Python help() function work well with decorators?

The following wrapper makes the cached function look more like the original.

from functools import wraps

def restore(func):
    @wraps(func.__wrapped__)
    def wrapper(*args, **kwargs):
        return func(*args, **kwargs)

    return wrapper

It creates yet another wrapper around your decorated function that restores its type as a function while preserving the docstring.

For example, if you have a function like this:

@restore
@lru_cache
def func_dup(x: int):
    """my doc"""
    return x

Then, run help(func_dup)

Help on function func_dup in module __main__:

func_dup(x: int)
    my doc

Why the Difference

I will be using CPython 3.10, which is the latest version as of the time I wrote this answer.

The help callable is actually implemented in pydoc as a Helper object. The magic method Helper.__call__ is defined to call Helper.help. It then calls doc, which calls render_doc. The render_doc function makes up the string that gets printed. Inside this function, it calls pydoc.describe for a descriptive name for your function.

Your original mymodule.myfunction is a function, so describe returns in this branch.

if inspect.isfunction(thing):
        return 'function ' + thing.__name__

This gives "function myfunction".

However, after you decorate your function with @lru_cache, it becomes an instance of the built-in/extension type functools._lru_cache_wrapper. I am not sure why it is implemented this way, but the decorated function is not of type types.FunctionType anymore. So the describe(mymodule.myfunction) function returns on the last line after being decorated.

return type(thing).__name__

This returns "_lru_cache_wrapper".

The functools.update_wrapper function attempts to

Update a wrapper function to look like the wrapped function

It doesn't restore the wrapper as an instance of types.FunctionType. It does, however, reference the original function in the __wrapped__ attribute. Hence, we can use that to wrap your original function yet again.

Reference

There is a Python Issue bpo-46761 that may or may not relate to this issue.

when using functools.partial() to pre-supply arguments to a function, if you then call functools.update_wrapper() to update that partial object, inspect.signature() returns the original function's signature, not the wrapped function's signature.

It is primarily on functools.partial, which doesn't even preserve the wrapped function's signature.

Using class attributes to modify a docstring with a decorator in Python

Turns out that accessing class attributes from within a class is impossible, as the class has yet to be executed when the decorator is called. So the original goal - using a decorator within a class to access class attributes - does not seem to be possible.

However, thanks to jdehesa for pointing me to a workaround that allows access to the class attributes using a class decorator, here: Can a Python decorator of an instance method access the class?.

I was able to use the class decorator to alter the specific method's docstring using class attributes like so:

def class_decorator(cls):
    for name, method in cls.__dict__.items():
        if name == 'example':
            # do something with the method
            method.__doc__ = "{} is new for function {} in class {}".format(method.__doc__, name, cls.__name__)
            # Note that other class attributes such as cls.__base__ 
            # can also be accessed in this way
    return cls

@class_decorator
class Test():
    def example(self, examplearg=1):
        """Docstring"""

print(Test().example.__doc__)
# Returns "Docstring is new for function example in class Test"

Should a docstring go before or after a decorator?

Where should the docstring go?

The docstring should go inside the function, the first thing after the function header:

@app.route("/")
def hello() -> str:
    """
    summary
    """
    return "Hello World"

The specification itself (PEP 257) makes it explicit:

A docstring is a string literal that occurs as the first statement in a module, function, class, or method definition.

Why?

It is important because docstrings are not just a convention.

If you put them in the right place, you can see the function documentation with the help() function (and maybe even other tools):

>>> @app.route("/")
... def hello() -> str:
...     """
...     summary
...     """
...     return "Hello World"
... 
>>> help(hello)
Help on function hello in module __main__:

hello() -> str
    summary

This happens because, if a string literal is the first thing in the function declaration, the interpreter will set it to the __doc__ attribute of the function:

>>> hello.__doc__
'\n    summary\n    '

help() basically just displays, nicely formatted, the value of the __doc__ attribute.

Python - using decorator.py to preserve a method docstring

Basic Descriptor/Decorator

You just need to keep in mind which function you should decorate. Your function is being created in __get__, so it won't help to use the wrapper as a decorator, instead, you need to apply it in the __get__ method. As an aside, you can use either functools.update_wrapper or decorators.decorator for this. They work very similarly, except that you have to keep the result of decorators.decorator whereas functools.update_wrapper returns None. Both have signature f(wrapper, wrapped).

from functools import update_wrapper
class class_or_instance(object):
    def __init__(self, fn):
        self.fn = fn

    def __get__(self, obj, cls):
        if obj is not None:
            f = lambda *args, **kwds: self.fn(obj, *args, **kwds)
        else:
            f = lambda *args, **kwds: self.fn(cls, *args, **kwds)
        # update the function to have the correct metadata
        update_wrapper(f, self.fn)
        return f

class A(object):
    @class_or_instance
    def func1(self,*args):
        """some docstring"""
        pass

Now if you do:

print A.func1.__doc__

You'll see "some docstring". Yay!

---

Cached property decorator

The key here is that you can only affect what gets returned. Since class_or_instance doesn't actually serve as the function, it doesn't really matter what you do with it. Keep in mind that this method causes the function to be rebound every time. I suggest you add a little bit of magic instead and bind/cache the function after the first call, which really just involves adding a setattr call.

from functools import update_wrapper
import types

class class_or_instance(object):
    # having optional func in case is passed something that doesn't have a correct __name__
    # (like a lambda function)
    def __init__(self, name_or_func):
        self.fn = fn
        self.name = fn.__name__

    def __get__(self, obj, cls):
        print "GET!!!"
        if obj is not None:
            f = lambda *args, **kwds: self.fn(obj, *args, **kwds)
            update_wrapper(f, self.fn)
            setattr(obj, self.name, types.MethodType(f, obj, obj.__class__))
        else:
            f = lambda *args, **kwds: self.fn(cls, *args, **kwds)
            update_wrapper(f, self.fn)
        return f

And then we can test it out...neato:

A.func1 #GET!!!
obj = A()
obj.func1 #GET!!!
obj.func1 is obj.func1 # True
A.func1 # GET!!!
obj2 = A()
obj2.func1 is not obj.fun1 # True + GET!!!

How to preserve a docstring of a decorated class for sphinx documentation?

If @wraps is not working you can update __doc__ manually.

Do something like:

def decorator(cls):
    class Wrapper(object):
        original = cls

        def __init__(self):
            self.__doc__ = self.original.__doc__
            self.__name__ = self.original.__name__

        def do_something_with_cls(cls):
            pass

    Wrapper.__doc__ = cls.__doc__
    return Wrapper

@decorator
class Foo(object):
    """The docstring I want to preserve."""
    def __init__(self):
        pass

print(Foo.__doc__)

'The docstring I want to preserve.'

---

Related Topics