Absolute VS. Explicit Relative Import of Python Module

Absolute vs. explicit relative import of Python module

Absolute imports. From PEP 8:

Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 [7] is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.

Explicit relative imports are a nice language feature (I guess), but they're not nearly as explicit as absolute imports. The more readable form is:

import A.A
import A.B.B

especially if you import several different namespaces. If you look at some well written projects/tutorials that include imports from within packages, they usually follow this style.

The few extra keystrokes you take to be more explicit will save others (and perhaps you) plenty of time in the future when they're trying to figure out your namespace (especially if you migrate to 3.x, in which some of the package names have changed).

Why do I have to use a relative import in a package's __init__.py?

When you run a python script, it's parent folder is added to sys.path

• run main.py: sys.path[0] = '../test'
• run init.py: sys.path[0] = '../test/pkg'

Your case: You try to "absolute-like" import a in __init__.py but the parent folder of a.py - which is '../test/pkg' - is not in the sys.path when you run main.py. This is why you get an error. However, your absolute import is incomplete as it should always start at the top level folder, e.g.

from test.pkg import a

Final answer to your question: You don't have to use relative imports!

See: PEP-8: Absolute imports are recommended, as they are usually more readable and tend to be better behaved (or at least give better error messages) if the import system is incorrectly configured (such as when a directory inside a package ends up on sys.path).

And keep in mind that relative imports don't work in a top-level-script when __name__ = "__main__", but from imported modules only.

You can learn more about absolute and relative imports here:

Absolute vs. explicit relative import of Python module

https://realpython.com/absolute-vs-relative-python-imports/

Absolute and relative import of Python modules: a matplotlib example

An important thing to remember about the matplotlib code base is that it is very old (have git history back to 2003 and have lost another few years), large (>93k lines of python, 17k lines of c++), and has over 450 contributors.

Having a look at git-blame (off of the 2.x branch but the imports are pretty stable) shows:

08:29 $ git blame matplotlib/artist.py | head -n 18

5fca7e31 (Thomas A Caswell         2013-09-25 11:36:00 -0500    1) from __future__ import (absolute_import, division, print_function,
5fca7e31 (Thomas A Caswell         2013-09-25 11:36:00 -0500    2)                         unicode_literals)
f4adec7b (Michael Droettboom       2013-08-14 10:18:10 -0400    3) 
07e22753 (Matthew Brett            2016-06-06 12:08:35 -0700    4) import six
0ea5fff3 (Thomas A Caswell         2015-12-01 14:40:34 -0500    5) from collections import OrderedDict
f4adec7b (Michael Droettboom       2013-08-14 10:18:10 -0400    6) 
453e0ece (Nelle Varoquaux          2012-08-27 23:16:43 +0200    7) import re
453e0ece (Nelle Varoquaux          2012-08-27 23:16:43 +0200    8) import warnings
731f6c86 (Michael Droettboom       2013-09-27 09:59:48 -0400    9) import inspect
e1d30c85 (Jens Hedegaard Nielsen   2015-08-18 19:52:48 +0100   10) import numpy as np
b44e8f20 (John Hunter              2008-12-08 23:28:55 +0000   11) import matplotlib
99b89a87 (John Hunter              2008-06-03 20:28:14 +0000   12) import matplotlib.cbook as cbook
c137a718 (Thomas A Caswell         2014-11-23 00:37:28 -0500   13) from matplotlib.cbook import mplDeprecation
527b7d9a (Michael Droettboom       2010-06-11 18:17:52 +0000   14) from matplotlib import docstring, rcParams
b2408c33 (Cimarron Mittelsteadt    2014-09-12 15:58:25 -0700   15) from .transforms import (Bbox, IdentityTransform, TransformedBbox,
b2408c33 (Cimarron Mittelsteadt    2014-09-12 15:58:25 -0700   16)                          TransformedPath, Transform)
f4adec7b (Michael Droettboom       2013-08-14 10:18:10 -0400   17) from .path import Path
f2a0c7ae (John Hunter              2007-03-20 21:48:31 +0000   18) 

You can see that these lines were last touched by a number of people (apparently including me) over a range of years.

I would not read too much into this difference, but if you want to dive deeper try looking at the commit messages on those changes.

Absolute vs Relative Imports in Python 2

To make these different types of import you have to put both <root>/ and <root>/test_package/ into sys.path, from each python file you want to execute. So in main it would be:

import os
import sys
import inspect
# Get the current folder, which is the input folder
current_folder = os.path.realpath(
    os.path.abspath(
        os.path.split(
            inspect.getfile(
                inspect.currentframe()
            )
     )[0]
   )
)
folder_parts = current_folder.split(os.sep)
previous_folder = os.sep.join(folder_parts[0:-1])

sys.path.insert(0, current_folder)
sys.path.insert(0, previous_folder)

# Rest of main

And in module B it would be:

import os
import sys
import inspect

from pprint import pprint
# Get the current folder, which is the input folder
mod_b_folder = os.path.realpath(
    os.path.abspath(
        os.path.split(
            inspect.getfile(
                inspect.currentframe()
            )
     )[0]
   )
)
folder_parts = mod_b_folder.split(os.sep)
prev_test_pack_path = os.sep.join(folder_parts[0:-2])
test_pack_path = os.sep.join(folder_parts[0:-1])
sys.path.insert(0, test_pack_path)
sys.path.insert(0, prev_test_pack_path)

# Rest of module B

However, I would recommend to use one naming system for importing modules and insert the appropriate folder into sys.path.

Relative imports for the billionth time

Script vs. Module

Here's an explanation. The short version is that there is a big difference between directly running a Python file, and importing that file from somewhere else. Just knowing what directory a file is in does not determine what package Python thinks it is in. That depends, additionally, on how you load the file into Python (by running or by importing).

There are two ways to load a Python file: as the top-level script, or as a
module. A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py on the command line. It is loaded as a module when an import statement is encountered inside some other file. There can only be one top-level script at a time; the top-level script is the Python file you ran to start things off.

Naming

When a file is loaded, it is given a name (which is stored in its __name__ attribute).

• If it was loaded as the top-level script, its name is __main__.
• If it was loaded as a module, its name is [ the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots ], for example, package.subpackage1.moduleX.

But be aware, if you load moduleX as a module from shell command line using something like python -m package.subpackage1.moduleX, the __name__ will still be __main__.

So for instance in your example:

package/
    __init__.py
    subpackage1/
        __init__.py
        moduleX.py
    moduleA.py

if you imported moduleX (note: imported, not directly executed), its name would be package.subpackage1.moduleX. If you imported moduleA, its name would be package.moduleA. However, if you directly run moduleX from the command line, its name will instead be __main__, and if you directly run moduleA from the command line, its name will be __main__. When a module is run as the top-level script, it loses its normal name and its name is instead __main__.

Accessing a module NOT through its containing package

There is an additional wrinkle: the module's name depends on whether it was imported "directly" from the directory it is in or imported via a package. This only makes a difference if you run Python in a directory, and try to import a file in that same directory (or a subdirectory of it). For instance, if you start the Python interpreter in the directory package/subpackage1 and then do import moduleX, the name of moduleX will just be moduleX, and not package.subpackage1.moduleX. This is because Python adds the current directory to its search path when the interpreter is entered interactively; if it finds the to-be-imported module in the current directory, it will not know that that directory is part of a package, and the package information will not become part of the module's name.

A special case is if you run the interpreter interactively (e.g., just type python and start entering Python code on the fly). In this case, the name of that interactive session is __main__.

Now here is the crucial thing for your error message: if a module's name has no dots, it is not considered to be part of a package. It doesn't matter where the file actually is on disk. All that matters is what its name is, and its name depends on how you loaded it.

Now look at the quote you included in your question:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top-level module, regardless of where the module is actually located on the file system.

Relative imports...

Relative imports use the module's name to determine where it is in a package. When you use a relative import like from .. import foo, the dots indicate to step up some number of levels in the package hierarchy. For instance, if your current module's name is package.subpackage1.moduleX, then ..moduleA would mean package.moduleA. For a from .. import to work, the module's name must have at least as many dots as there are in the import statement.

... are only relative in a package

However, if your module's name is __main__, it is not considered to be in a package. Its name has no dots, and therefore you cannot use from .. import statements inside it. If you try to do so, you will get the "relative-import in non-package" error.

Scripts can't import relative

What you probably did is you tried to run moduleX or the like from the command line. When you did this, its name was set to __main__, which means that relative imports within it will fail, because its name does not reveal that it is in a package. Note that this will also happen if you run Python from the same directory where a module is, and then try to import that module, because, as described above, Python will find the module in the current directory "too early" without realizing it is part of a package.

Also remember that when you run the interactive interpreter, the "name" of that interactive session is always __main__. Thus you cannot do relative imports directly from an interactive session. Relative imports are only for use within module files.

Two solutions:

1. If you really do want to run moduleX directly, but you still want it to be considered part of a package, you can do python -m package.subpackage1.moduleX. The -m tells Python to load it as a module, not as the top-level script.

2. Or perhaps you don't actually want to run moduleX, you just want to run some other script, say myfile.py, that uses functions inside moduleX. If that is the case, put myfile.py somewhere else – not inside the package directory – and run it. If inside myfile.py you do things like from package.moduleA import spam, it will work fine.

Notes

• For either of these solutions, the package directory (package in your example) must be accessible from the Python module search path (sys.path). If it is not, you will not be able to use anything in the package reliably at all.

• Since Python 2.6, the module's "name" for package-resolution purposes is determined not just by its __name__ attributes but also by the __package__ attribute. That's why I'm avoiding using the explicit symbol __name__ to refer to the module's "name". Since Python 2.6 a module's "name" is effectively __package__ + '.' + __name__, or just __name__ if __package__ is None.)

When to use absolute imports

Relative imports turned out to be a very bad idea, even though they were the default behavior for long. You can find quite a few questions on this site where someone simply named their file after a builtin module and broke their application with weird error messages.

That's why it's always a good to do absolute imports by referencing your project everywhere, including packages.

In short, use this style:

import myproject.mypackage
from myproject.mypackage.myfile import MyClass

Quote from PEP8:

Relative imports for intra-package imports are highly
discouraged.
Always use the absolute package path for all imports.
Even now that PEP 328 is fully implemented in Python 2.5,
its style of explicit relative imports is actively discouraged;
absolute imports are more portable and usually more readable.

Google Python Style Guide & relative imports

This is a reference to the Python 2 behavior (deprecated since 2.6) of implicit relative imports: allowing import bar in a module in a package foo to refer to the module foo.bar. Consider a directory on sys.path that looks like

…
|-- client.py
`-- pkg
    |-- __init__.py
    |-- mod.py
    `-- script.py

with files having the following contents:

client.py

print "client..."
from pkg import mod,script
print "client!"

pkg/__init__.py

print "pkg"

pkg/mod.py

print "mod: %r"%__name__

pkg/script.py

print "script:",__name__,__package__

if __name__=='__main__':
  import mod,client
  print "script!"

In this setup mod can easily be imported twice:

$ PYTHONPATH=… python …/pkg/script.py
script: __main__ None
mod: 'mod'
client...
pkg
mod: 'pkg.mod'
script: pkg.script None
client!
script!

In an attempt to reduce configuration overhead, Python adds the directory pkg to sys.path, effectively presuming that script.py is a top-level module script. Unfortunately, that means that import mod creates a top-level module named mod, and the explicit import of pkg.mod later causes another copy of it to exist with its full name (just after importing pkg itself).

It was recognized that this poses a problem, and later -m was adjusted to tell the module being executed about the package in which it was found so that relative imports (implicit or explicit) work properly:

$ PYTHONPATH=… python -m pkg.script
pkg
script: __main__ pkg
mod: 'pkg.mod'
client...
script: pkg.script None
client!
script!

Note that pkg is now imported first (by -m itself!), that script now has a __package__ attribute immediately afterwards, and that mod is imported just once. Of course, script itself is (still) loaded twice, since its name gets replaced with __main__ the first time so that from pkg import script finds it under a different name.

The moral is that implementing "a module can also be a script" in terms of __name__=='__main__' is fundamentally broken (and replacements have been rejected): a module already has a __name__, and creating a separate module object to be the entry point so that its __name__ can differ is as absurd as copying the Java class (and all its static data) that provides main. Making a module that no one ever imports works, but is oxymoronic (and breaks code inspection that imports all members of a package).

---

Related Topics