What Happens When Using Mutual or Circular (Cyclic) Imports in Python

What happens when using mutual or circular (cyclic) imports in Python?

There was a really good discussion on this over at comp.lang.python last year. It answers your question pretty thoroughly.

Imports are pretty straightforward really. Just remember the following:

'import' and 'from xxx import yyy' are executable statements. They execute
when the running program reaches that line.

If a module is not in sys.modules, then an import creates the new module
entry in sys.modules and then executes the code in the module. It does not
return control to the calling module until the execution has completed.

If a module does exist in sys.modules then an import simply returns that
module whether or not it has completed executing. That is the reason why
cyclic imports may return modules which appear to be partly empty.

Finally, the executing script runs in a module named __main__, importing
the script under its own name will create a new module unrelated to
__main__.

Take that lot together and you shouldn't get any surprises when importing
modules.

How to avoid circular imports in Python?

Only import the module, don't import from the module:

Consider a.py:

import b

class A:
    def bar(self):
        return b.B()

and b.py:

import a

class B:
    def bar(self):
        return a.A()

This works perfectly fine.

Why do circular imports seemingly work further up in the call stack but then raise an ImportError further down?

I think the answer by jpmc26, while by no means wrong, comes down too heavily on circular imports. They can work just fine, if you set them up correctly.

The easiest way to do so is to use import my_module syntax, rather than from my_module import some_object. The former will almost always work, even if my_module included imports us back. The latter only works if my_object is already defined in my_module, which in a circular import may not be the case.

To be specific to your case: Try changing entities/post.py to do import physics and then refer to physics.PostBody rather than just PostBody directly. Similarly, change physics.py to do import entities.post and then use entities.post.Post rather than just Post.

Circular import dependency in Python

If a depends on c and c depends on a, aren't they actually the same unit then?

You should really examine why you have split a and c into two packages, because either you have some code you should split off into another package (to make them both depend on that new package, but not each other), or you should merge them into one package.

Circular imports Python

Here's a simplified sequence of events that happen in the code in Python 3:

1. __init__.py starts running
   
   • An empty __main__ module is added to sys.modules
2. import module1 starts loading module1.py
   
   • An empty module1 module is added to sys.modules
3. import module2 starts loading module2.py
   
   • An empty module2 module is added to sys.modules
4. module2.function2 is created and added to module2.__dict__
   
   • The fact that function2 references names in module1 does not affect the creation of the function object in any way
5. module2 is fully loaded and execution returns to module1
6. module1.function1 and module1.function3 are created and added to module1.__dict__
   
   • Again, it does not matter what names the functions reference because they are not being called. AttributeError and NameError can be raised at runtime if necessary.
7. module1 is fully loaded and execution returns to __main__
8. module1.function runs successfully, since all the names it references are resolvable.

As you can see, there are no circular import issues in this particular sequence of imports because module1 and module2 do not attempt to call each other's functions. The current import system allows both modules to load before the functions are called.

The post you mention is from 2017, and must be using a version of python from before 3.0. A hint is found in the link in the following quote, which links to the python-2.x docs:

This approach doesn't contradict Python syntax, as the Python documentation says: "It is customary but not required to place all import statements at the beginning of a module (or script, for that matter)".

The paragraph after that is a bit misleading by the way:

The Python documentation also says that it is advisable to use import X, instead of other statements, such as from module import *, or from module import a,b,c.

While star imports are certainly discouraged, specific-name imports of the form from module import a,b,c are generally very much encouraged with few exceptions.

Avoiding circular (cyclic) imports in Python?

Merge any pair of modules that depend on each other into a single module. Then introduce extra modules to get the old names back.

E.g.,

# a.py
from b import B

class A: whatever

# b.py
from a import A

class B: whatever

becomes

# common.py
class A: whatever
class B: whatever

# a.py
from common import A

# b.py
from common import B

Understanding behavior of Python imports and circular dependencies

When Python starts loading the pkg.a module, it sets sys.modules['pkg.a'] to the corresponding module object, but it only sets the a attribute of the pkg module object at the very end of loading the pkg.a module. This will be relevant later.

---

Relative imports are from imports, and they behave the same. After from . import whatever figures out that . refers to the pkg package, it goes ahead with the regular from pkg import whatever logic.

When c.py hits from . import a, first, it sees that pkg.a is already in sys.modules, indicating that pkg.a has already been loaded or is in the middle of being loaded. (It's in the middle of being loaded, but this code path doesn't care.) It skips to the second part of its job, retrieving pkg.a and assigning it to the a name in the local namespace, but it doesn't just retrieve sys.modules['pkg.a'] to do this.

You know how you can do stuff like from os import open, even though os.open is a function, not a module? That kind of import can't go through sys.modules['os.open'], because os.open isn't a module and isn't in sys.modules. Instead, all from imports, including all relative imports, attempt an attribute lookup on the module they're importing names from. from . import a looks up the a attribute on the pkg module object, but it's not there, because that attribute only gets set when pkg.a finishes loading.

On Python 2, that's it. End of import. ImportError here. On Python 3 (specifically 3.5+), because they wanted to encourage relative imports and this behavior is really inconvenient, from imports try one more step. If the attribute lookup fails, now they try sys.modules. pkg.a is in sys.modules, so the import succeeds. You can see the discussion for this change in the CPython issue tracker at issue 17636.

---

Related Topics