Why Might Python's 'From' Form of an Import Statement Bind a Module Name

Why might Python's `from` form of an import statement bind a module name?

From the import system documentation:

When a submodule is loaded using any mechanism (e.g. importlib APIs,
the import or import-from statements, or built-in __import__())
a binding is placed in the parent module’s namespace to the submodule
object. For example, if package spam has a submodule foo, after
importing spam.foo, spam will have an attribute foo which is
bound to the submodule. Let’s say you have the following directory
structure:

spam/
__init__.py
foo.py
bar.py

and spam/__init__.py has the following lines in it:

from .foo import Foo
from .bar import Bar

then executing the following puts a name binding to foo and bar in
the spam module:

>>> import spam
>>> spam.foo
<module 'spam.foo' from '/tmp/imports/spam/foo.py'>
>>> spam.bar
<module 'spam.bar' from '/tmp/imports/spam/bar.py'>

Given Python’s familiar name binding rules this might seem surprising,
but it’s actually a fundamental feature of the import system. The
invariant holding is that if you have sys.modules['spam'] and
sys.modules['spam.foo'] (as you would after the above import), the
latter must appear as the foo attribute of the former.

If you do from testapp.api.utils import x, the import statement will not load utils into the local namespace. However, the import machinery will load utils into the testapp.api namespace, to make further imports work right. It just happens that in your case, testapp.api is also the local namespace, so you're getting a surprise.

Python import statement without a from clause imports a class not a module

Updated:

This is a namespace collision between the module binding a in pkg and the variable binding a in pkg.
This is expected behaviour as described here

Old:

I believe this is because of the finder used by the import statement.
The finder will search the package first before the module path, and because it finds a path that matches it immediately returns.

I took a look at the python import docs here and found this little snippet

When the name variable is of the form package.module, normally, the top-level package (the name up till the first dot) is returned, not the module named by name. However, when a non-empty fromlist argument is given, the module named by name is returned.

If you want to force a module import you can use the import_module function

import importlib
a = importlib.import_module('pkg.a')

Why does Python 2 try to get a module as a package attribute when using import ... as ...?

If you do

import package.submodule

and then try to access package.submodule, that access is an attribute lookup on the module object for package, and it will find whatever object is bound to the submodule attribute (or fail if that attribute is unset). For consistency,

import package.submodule as whatever

performs the same attribute lookup to find the object to bind to whatever.

This was changed in Python 3.7 to fall back to a sys.modules['package.submodule'] lookup if the attribute lookup fails. This change was made for consistency with a previous change in Python 3.5 that made from package import submodule fall back to a sys.modules lookup, and that change was made to make relative imports more convenient.

from ... import OR import ... as for modules

Assuming that bar is a module or package in foo, there is no difference*, it doesn't matter. The two statements have exactly the same result:

>>> import os.path as path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>
>>> from os import path
>>> path
<module 'posixpath' from '/Users/mj/Development/venvs/stackoverflow-2.7/lib/python2.7/posixpath.pyc'>

If bar is not a module or package, the second form will not work; a traceback is thrown instead:

>>> import os.walk as walk
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named walk

* In Python 3.6 and before, there was a bug with the initialization ordering of packages containing other modules, where in the loading stage of the package using import contained.module.something as alias in a submodule would fail where from contained.module import something as alias would not. See Imports in __init__.py and `import as` statement for a very illustrative example of that problem, as well as Python issues #23203 and #30024.

Use 'import module' or 'from module import'?

The difference between import module and from module import foo is mainly subjective. Pick the one you like best and be consistent in your use of it. Here are some points to help you decide.

import module

  • Pros:
    • Less maintenance of your import statements. Don't need to add any additional imports to start using another item from the module
  • Cons:
    • Typing module.foo in your code can be tedious and redundant (tedium can be minimized by using import module as mo then typing mo.foo)

from module import foo

  • Pros:
    • Less typing to use foo
    • More control over which items of a module can be accessed
  • Cons:
    • To use a new item from the module you have to update your import statement
    • You lose context about foo. For example, it's less clear what ceil() does compared to math.ceil()

Either method is acceptable, but don't use from module import *.

For any reasonable large set of code, if you import * you will likely be cementing it into the module, unable to be removed. This is because it is difficult to determine what items used in the code are coming from 'module', making it easy to get to the point where you think you don't use the import any more but it's extremely difficult to be sure.

Why does __init__.py import all files in the folder as modules?

Why does __init__.py import all files in the folder as modules?

It does not. You must explicitly import them. Add an empty other.py file in the package directory and you will not see it does not get imported at all, unless you also add the code to trigger the submodule import. See Do I need to import submodules directly? for more about that.

You've probably meant to ask something more like "Why do submodule imports always bind names in the parent module?" as a question title. Well, this is just how the import system in Python works. It's documented under 5.4.2. Submodules:

When a submodule is loaded using any mechanism (e.g. importlib APIs, the import or import-from statements, or built-in __import__()) a binding is placed in the parent module’s namespace to the submodule object. For example, if package spam has a submodule foo, after importing spam.foo, spam will have an attribute foo which is bound to the submodule.

To give you a better intuition about why this is actually a sensible behavior, consider that importing the parent module (my_module) creates an item in sys.modules dict, with the key being "my_module" (string) and the value being my_module (module). Subsequently, the submodule import of client will also create an item in sys.modules with the key being "my_module.client" (string) and the value being my_module.client (module). Since both my_module and my_module.client are now cached in the sys.modules, it makes sense for client to be attached as an attribute on my_module.

If you don't like the submodule name bindings appearing for whatever reason, you may simply remove those submodule names from the parent module namespace by placing a del statement after the import-from statements.

`from ... import` vs `import .`

It depends on how you want to access the import when you refer to it.

from urllib import request
# access request directly.
mine = request()

import urllib.request
# used as urllib.request
mine = urllib.request()

You can also alias things yourself when you import for simplicity or to avoid masking built ins:

from os import open as open_
# lets you use os.open without destroying the
# built in open() which returns file handles.

Python imports - why main package name is required?

Absolute imports, like import x.y.z or from x.y import z require x to be in your path. In your specific case, myModel is on the path because of your working directory. The sub-packages are not on the path, and can therefore only be accessed by reiterating the root package.

A more intuitive approach might be to use relative paths. This is possible because all your files live in proper packages with __init__ files. Keep in mind that relative paths imply that you have modules that are designed to live in your package structure and not on their own. Otherwise, you may end up causing errors when you try to run some of the modules as standalone scripts.

Change myModel/__init__.py to:

from . import models
from . import loader

The . makes the import relative. Notice that I did not suggest changing main.py, since it lives outside your packages. Adding more dots lets you go up more levels in the file hierarchy.

Change myModel/loader/__init__.py to

from .dataset_loader import *

and myModel/models/__init__.py to

from .static_model import StaticModel

An import statement binds a name in your local namespace (usually the module you are executing it in). The name that is bound depends on which form of import you used:

  1. import x binds the module described in x.py or x/__init__.py to the name x
  2. import x.y binds the module described in x.py or x/__init__.py to the name x, and insures that x has an attribute y, either as an attribute defined in x.py/__init__.py, or as a sub-module in x/y.py.
  3. from x import y binds the attribute or sub-module y from x.py/x/__init__.py or x/y.py to the name y. This option loads, but does not give you access to x.

When you run from myModel import loader, you get a module object loader that has a callable attribute fun.



Related Topics



Leave a reply



Submit