How to Implement _Getattribute_ Without an Infinite Recursion Error

How do I implement __getattribute__ without an infinite recursion error?

You get a recursion error because your attempt to access the self.__dict__ attribute inside __getattribute__ invokes your __getattribute__ again. If you use object's __getattribute__ instead, it works:

class D(object):
    def __init__(self):
        self.test=20
        self.test2=21
    def __getattribute__(self,name):
        if name=='test':
            return 0.
        else:
            return object.__getattribute__(self, name)

This works because object (in this example) is the base class. By calling the base version of __getattribute__ you avoid the recursive hell you were in before.

Ipython output with code in foo.py:

In [1]: from foo import *

In [2]: d = D()

In [3]: d.test
Out[3]: 0.0

In [4]: d.test2
Out[4]: 21

Update:

There's something in the section titled More attribute access for new-style classes in the current documentation, where they recommend doing exactly this to avoid the infinite recursion.

Avoiding infinite loops in __getattribute__

You seem to be under the impression that your implementation of __getattribute__ is merely a hook, that if you provide it Python will call it, and otherwise the interpreter will do its normal magic directly.

That is not correct. When python looks up attributes on instances, __getattribute__ is the main entry for all attribute access, and object provides the default implementation. Your implementation is thus overriding the original, and if your implementation provides no alternative means of returning attributes it fails. You cannot use attribute access in that method, since all attribute access to the instance (self) is channelled again through type(self).__getattribute__(self, attr).

The best way around this is by calling the overridden original again. That's where super(C, self).__getattribute__(attr) comes in; you are asking the next class in the class-resolution order to take care of the attribute access for you.

Alternatively, you can call the unbound object.__getattribute__() method directly. The C implementation of this method is the final stop for attribute access (it has direct access to __dict__ and is thus not bound to the same limitations).

Note that super() returns a proxy object that'll look up whatever method can be found next in the method-resolution ordered base classes. If no such method exists, it'll fail with an attribute error. It will never call the original method. Thus Foo.bar() looking up super(Foo, self).bar will either be a base-class implementation or an attribute error, never Foo.bar itself.

python __getattribute__ RecursionError when returning class variable attribute

The __getattribute__ method is called unconditionally to look up all attributes on an object, not only ones that don't exist (which is what __getattr__ does). When you do self.__foo in its implementation, you recurse, since __foo is another attribute that we're trying to look up on the object.

To avoid this issue, you need to call your parent's __getattribute__ method to get all of your own attributes inside the __getattribute__ method:

def __getattribute__(self, attr):
    try:
        return getattr(super().__getattribute__("_Foo__foo"), attr)
    except AttributeError:
        super().__getattribute__(attr)

Note that I had to manually apply name mangling to the __foo attribute, because we need to pass the name as a string to super().__getattribute__. That probably suggests you shouldn't be doing the mangling in the first place. A name with a single leading underscore might be a better choice.

Why there is infinite recursion loop risk in __getattribute__?

I will try to make it simpler by breaking the self.__dict__[item] into 2 parts:

class Count(object):
    def __getattr__(self, item):
        print('__getattr__:', item)
        d = self.__dict__
        print('resolved __dict__')
        d[item] = 0
        return 0

    def __getattribute__(self, item):
        print('__getattribute__:', item)
        if item.startswith('cur'):
            raise AttributeError
        return super(Count, self).__getattribute__(item)

obj1 = Count()
print(obj1.current)

The output is

__getattribute__: current
__getattr__: current
__getattribute__: __dict__
resolved __dict__
0

Now, if we replace super(Count, self) with the incorrect construct super(object, self) the message is not printed. It is because __getattribute__ will also mask the access to __dict__. However the super object will point to the base class of object which does not exist and hence our __getattribute__ function will always throw AttributeError.

Now, after __getattribute__ fails, __getattr__ is being tried for it ... well, instead of just resolving __dict__ to some value, it tries to get it as an attribute - and ends up calling__getattribute__ again. Hence we get.

....
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
Traceback (most recent call last):
  File "getattribute.py", line 15, in <module>
    print(obj1.current)
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  [Previous line repeated 328 more times]
  File "getattribute.py", line 8, in __getattribute__
    print('__getattribute__: ', item)
RecursionError: maximum recursion depth exceeded while calling a Python object

Had you used setattr(self, item, 0) instead of looking up self.__dict__ this could have been "avoided":

class Count(object):
    def __getattr__(self, item):
        setattr(self, item, 0)
        return 0

    def __getattribute__(self, item):
        if item.startswith('cur'):
            raise AttributeError
        return super(object, self).__getattribute__(item)

obj1 = Count()
print(obj1.current)

of course such code would not have been correct - trying to access any other attribute would have failed nevertheless.

How to override getattr or getattribute in python to call gettattr or getattribute inside itself as in lazy recursive directory listing

The easiest method is to implement __getattr__, which is only called if a normal lookup fails. That means that you can access normal attributes inside your __getattr__ implementation without having to do anything special, and it won't be called recursively because the base class lookup succeeds.

    def __getattr__(self, name):
        return self._list[0]  # works fine!

__getattribute__ is more of a special case because it's called unconditionally rather than as a fallback to the normal lookup. Per the docs (https://docs.python.org/3/reference/datamodel.html#object.getattribute)
if you want to access an attribute inside __getattribute__ without calling your own overridden __getattribute__ implementation you need to call the object implementation:

    def __getattribute__(self, name):
        return object.__getattribute__(self, '_list')[0]

Format class attributes with __getattribute__ overriding

You got an infinite recursion when you try to access self.VERSION here f"/api/{route}/v{self.VERSION}" because self.VERSION also invokes __getattribute__ method

So, you should handle VERSION attribute separately

from dataclasses import dataclass

@dataclass
class _BaseRouteRegistry:
    VERSION = '1'
    USERS = 'users'

    def __getattribute__(self, item):
        if item == "VERSION":
            return object.__getattribute__(self, item)

        route = object.__getattribute__(self, item)
        return f"/api/{route}/v{self.VERSION}"

BaseRouteRegistry = _BaseRouteRegistry()

print(BaseRouteRegistry.USERS)

> /api/users/v1

Infinite recursion for __getattr__() - but why is it even called once?

copy.copy() tries to access the __setstate__ attribute on an empty instance, no attributes set yet, because __init__ hasn't been called (and won't be; copy.copy() is responsible to set the new attributes on it).

The __setstate__ method copy.copy() is looking for is optional, and your class indeed has no such attribute. Because it is missing, __getattr__ is called for it, and because there is no __all_parts attribute yet either, the for i in self.__all_parts: line ends up calling __getattr__ again. And again and again and again.

The trick is to cut out of this loop early by testing for it. The best way to do this is to special-case all attributes that start with an underscore:

if item.startswith('_'):
    # bail out early
    raise AttributeError(item)

---

Related Topics