Circular Dependency in Python

Circular dependency in Python

Importing Python Modules is a great article that explains circular imports in Python.

The easiest way to fix this is to move the path import to the end of the node module.

How to avoid circular imports in Python?

Only import the module, don't import from the module:

Consider a.py:

import b

class A:
    def bar(self):
        return b.B()

and b.py:

import a

class B:
    def bar(self):
        return a.A()

This works perfectly fine.

What happens when using mutual or circular (cyclic) imports in Python?

There was a really good discussion on this over at comp.lang.python last year. It answers your question pretty thoroughly.

Imports are pretty straightforward really. Just remember the following:

'import' and 'from xxx import yyy' are executable statements. They execute
when the running program reaches that line.

If a module is not in sys.modules, then an import creates the new module
entry in sys.modules and then executes the code in the module. It does not
return control to the calling module until the execution has completed.

If a module does exist in sys.modules then an import simply returns that
module whether or not it has completed executing. That is the reason why
cyclic imports may return modules which appear to be partly empty.

Finally, the executing script runs in a module named __main__, importing
the script under its own name will create a new module unrelated to
__main__.

Take that lot together and you shouldn't get any surprises when importing
modules.

Understanding behavior of Python imports and circular dependencies

When Python starts loading the pkg.a module, it sets sys.modules['pkg.a'] to the corresponding module object, but it only sets the a attribute of the pkg module object at the very end of loading the pkg.a module. This will be relevant later.

---

Relative imports are from imports, and they behave the same. After from . import whatever figures out that . refers to the pkg package, it goes ahead with the regular from pkg import whatever logic.

When c.py hits from . import a, first, it sees that pkg.a is already in sys.modules, indicating that pkg.a has already been loaded or is in the middle of being loaded. (It's in the middle of being loaded, but this code path doesn't care.) It skips to the second part of its job, retrieving pkg.a and assigning it to the a name in the local namespace, but it doesn't just retrieve sys.modules['pkg.a'] to do this.

You know how you can do stuff like from os import open, even though os.open is a function, not a module? That kind of import can't go through sys.modules['os.open'], because os.open isn't a module and isn't in sys.modules. Instead, all from imports, including all relative imports, attempt an attribute lookup on the module they're importing names from. from . import a looks up the a attribute on the pkg module object, but it's not there, because that attribute only gets set when pkg.a finishes loading.

On Python 2, that's it. End of import. ImportError here. On Python 3 (specifically 3.5+), because they wanted to encourage relative imports and this behavior is really inconvenient, from imports try one more step. If the attribute lookup fails, now they try sys.modules. pkg.a is in sys.modules, so the import succeeds. You can see the discussion for this change in the CPython issue tracker at issue 17636.

Python circular dependency with the inclusion of member functions

It sounds like you've caused a lot of trouble for yourself by having primitive and square defined in separate modules/files. If you defined them in the same module, I doubt you'd have any trouble. For example, the following works fine:

class Primitive:
    pass

class Square(Primitive):
    def union(self, other):
        return Union(self, other)

class Union(Primitive):
    def __init__(self, *members):
        self.members = members

obj1 = Square()
obj2 = Square()
obj3 = obj1.union(obj2)

print(type(obj3))
print(obj3.members)

If you insist on putting your classes in different files, though, you can do something like this:

primitive.py:

    class Primitive:
        pass

square.py:

    from .primitive import Primitive

    class Square(Primitive):
        def union(self, other):
            from .union import Union
            return Union(self, other)

union.py:

    from .primitive import Primitive

    class Union(Primitive):
        def __init__(self, *members):
            self.members = members

test.py:

    from .square import Square

    obj1 = Square()
    obj2 = Square()
    obj3 = obj1.union(obj2)

    print(type(obj3))
    print(obj3.members)

The key point is moving the from .union import Union statement inside the union() method, where it won't be invoked until it's needed.

Here's a good resource on Python circular imports.

Type hints: solve circular dependency

You can use a forward reference by using a string name for the not-yet-defined Client class:

class Server():
    def register_client(self, client: 'Client')
        pass

As of Python 3.7, you can also postpone all runtime parsing of annotations by adding the following __future__ import at the top of your module:

from __future__ import annotations

at which point the annotations are stored as string representations of the abstract syntax tree for the expression; you can use typing.get_type_hints() to resolve those (and resolve forward references as used above).

See PEP 563 -- Postponed Evaluation of Annotations for details; this behaviour will be the default in Python 4.0.

---

Related Topics