Relative Imports in Python 3

Relative imports in Python 3

unfortunately, this module needs to be inside the package, and it also
needs to be runnable as a script, sometimes. Any idea how I could
achieve that?

It's quite common to have a layout like this...

main.py
mypackage/
    __init__.py
    mymodule.py
    myothermodule.py

...with a mymodule.py like this...

#!/usr/bin/env python3

# Exported function
def as_int(a):
    return int(a)

# Test function for module  
def _test():
    assert as_int('1') == 1

if __name__ == '__main__':
    _test()

...a myothermodule.py like this...

#!/usr/bin/env python3

from .mymodule import as_int

# Exported function
def add(a, b):
    return as_int(a) + as_int(b)

# Test function for module  
def _test():
    assert add('1', '1') == 2

if __name__ == '__main__':
    _test()

...and a main.py like this...

#!/usr/bin/env python3

from mypackage.myothermodule import add

def main():
    print(add('1', '1'))

if __name__ == '__main__':
    main()

...which works fine when you run main.py or mypackage/mymodule.py, but fails with mypackage/myothermodule.py, due to the relative import...

from .mymodule import as_int

The way you're supposed to run it is...

python3 -m mypackage.myothermodule

...but it's somewhat verbose, and doesn't mix well with a shebang line like #!/usr/bin/env python3.

The simplest fix for this case, assuming the name mymodule is globally unique, would be to avoid using relative imports, and just use...

from mymodule import as_int

...although, if it's not unique, or your package structure is more complex, you'll need to include the directory containing your package directory in PYTHONPATH, and do it like this...

from mypackage.mymodule import as_int

...or if you want it to work "out of the box", you can frob the PYTHONPATH in code first with this...

import sys
import os

SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
sys.path.append(os.path.dirname(SCRIPT_DIR))

from mypackage.mymodule import as_int

It's kind of a pain, but there's a clue as to why in an email written by a certain Guido van Rossum...

I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.

Whether running scripts inside a package is an antipattern or not is subjective, but personally I find it really useful in a package I have which contains some custom wxPython widgets, so I can run the script for any of the source files to display a wx.Frame containing only that widget for testing purposes.

How to do relative imports in Python?

Everyone seems to want to tell you what you should be doing rather than just answering the question.

The problem is that you're running the module as '__main__' by passing the mod1.py as an argument to the interpreter.

From PEP 328:

Relative imports use a module's __name__ attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to '__main__') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.

In Python 2.6, they're adding the ability to reference modules relative to the main module. PEP 366 describes the change.

Update: According to Nick Coghlan, the recommended alternative is to run the module inside the package using the -m switch.

Relative imports for the billionth time

Script vs. Module

Here's an explanation. The short version is that there is a big difference between directly running a Python file, and importing that file from somewhere else. Just knowing what directory a file is in does not determine what package Python thinks it is in. That depends, additionally, on how you load the file into Python (by running or by importing).

There are two ways to load a Python file: as the top-level script, or as a
module. A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py on the command line. It is loaded as a module when an import statement is encountered inside some other file. There can only be one top-level script at a time; the top-level script is the Python file you ran to start things off.

Naming

When a file is loaded, it is given a name (which is stored in its __name__ attribute).

If it was loaded as the top-level script, its name is __main__.
If it was loaded as a module, its name is [ the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots ], for example, package.subpackage1.moduleX.

But be aware, if you load moduleX as a module from shell command line using something like python -m package.subpackage1.moduleX, the __name__ will still be __main__.

So for instance in your example:

package/
    __init__.py
    subpackage1/
        __init__.py
        moduleX.py
    moduleA.py

if you imported moduleX (note: imported, not directly executed), its name would be package.subpackage1.moduleX. If you imported moduleA, its name would be package.moduleA. However, if you directly run moduleX from the command line, its name will instead be __main__, and if you directly run moduleA from the command line, its name will be __main__. When a module is run as the top-level script, it loses its normal name and its name is instead __main__.

Accessing a module NOT through its containing package

There is an additional wrinkle: the module's name depends on whether it was imported "directly" from the directory it is in or imported via a package. This only makes a difference if you run Python in a directory, and try to import a file in that same directory (or a subdirectory of it). For instance, if you start the Python interpreter in the directory package/subpackage1 and then do import moduleX, the name of moduleX will just be moduleX, and not package.subpackage1.moduleX. This is because Python adds the current directory to its search path when the interpreter is entered interactively; if it finds the to-be-imported module in the current directory, it will not know that that directory is part of a package, and the package information will not become part of the module's name.

A special case is if you run the interpreter interactively (e.g., just type python and start entering Python code on the fly). In this case, the name of that interactive session is __main__.

Now here is the crucial thing for your error message: if a module's name has no dots, it is not considered to be part of a package. It doesn't matter where the file actually is on disk. All that matters is what its name is, and its name depends on how you loaded it.

Now look at the quote you included in your question:

Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top-level module, regardless of where the module is actually located on the file system.

Relative imports...

Relative imports use the module's name to determine where it is in a package. When you use a relative import like from .. import foo, the dots indicate to step up some number of levels in the package hierarchy. For instance, if your current module's name is package.subpackage1.moduleX, then ..moduleA would mean package.moduleA. For a from .. import to work, the module's name must have at least as many dots as there are in the import statement.

... are only relative in a package

However, if your module's name is __main__, it is not considered to be in a package. Its name has no dots, and therefore you cannot use from .. import statements inside it. If you try to do so, you will get the "relative-import in non-package" error.

Scripts can't import relative

What you probably did is you tried to run moduleX or the like from the command line. When you did this, its name was set to __main__, which means that relative imports within it will fail, because its name does not reveal that it is in a package. Note that this will also happen if you run Python from the same directory where a module is, and then try to import that module, because, as described above, Python will find the module in the current directory "too early" without realizing it is part of a package.

Also remember that when you run the interactive interpreter, the "name" of that interactive session is always __main__. Thus you cannot do relative imports directly from an interactive session. Relative imports are only for use within module files.

Two solutions:

If you really do want to run moduleX directly, but you still want it to be considered part of a package, you can do python -m package.subpackage1.moduleX. The -m tells Python to load it as a module, not as the top-level script.
Or perhaps you don't actually want to run moduleX, you just want to run some other script, say myfile.py, that uses functions inside moduleX. If that is the case, put myfile.py somewhere else – not inside the package directory – and run it. If inside myfile.py you do things like from package.moduleA import spam, it will work fine.

Notes

For either of these solutions, the package directory (package in your example) must be accessible from the Python module search path (sys.path). If it is not, you will not be able to use anything in the package reliably at all.
Since Python 2.6, the module's "name" for package-resolution purposes is determined not just by its __name__ attributes but also by the __package__ attribute. That's why I'm avoiding using the explicit symbol __name__ to refer to the module's "name". Since Python 2.6 a module's "name" is effectively __package__ + '.' + __name__, or just __name__ if __package__ is None.)

How can I use relative importing in Python3 with an if name='main' block?

According to the Module documentation, for __main__ modules, you have to use absolute imports.

Note that relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application must always use absolute imports.

So just change the import line in module1.py to:

from mypackage import module2

Everything else remains the same.

Changes in import statement python3

Relative import happens whenever you are importing a package relative to the current script/package.

Consider the following tree for example:

mypkg
├── base.py
└── derived.py

Now, your derived.py requires something from base.py. In Python 2, you could do it like this (in derived.py):

from base import BaseThing

Python 3 no longer supports that since it's not explicit whether you want the 'relative' or 'absolute' base. In other words, if there was a Python package named base installed in the system, you'd get the wrong one.

Instead it requires you to use explicit imports which explicitly specify location of a module on a path-alike basis. Your derived.py would look like:

from .base import BaseThing

The leading . says 'import base from module directory'; in other words, .base maps to ./base.py.

Similarly, there is .. prefix which goes up the directory hierarchy like ../ (with ..mod mapping to ../mod.py), and then ... which goes two levels up (../../mod.py) and so on.

Please however note that the relative paths listed above were relative to directory where current module (derived.py) resides in, not the current working directory.

@BrenBarn has already explained the star import case. For completeness, I will have to say the same ;).

For example, you need to use a few math functions but you use them only in a single function. In Python 2 you were permitted to be semi-lazy:

def sin_degrees(x):
    from math import *
    return sin(degrees(x))

Note that it already triggers a warning in Python 2:

a.py:1: SyntaxWarning: import * only allowed at module level
  def sin_degrees(x):

In modern Python 2 code you should and in Python 3 you have to do either:

def sin_degrees(x):
    from math import sin, degrees
    return sin(degrees(x))

or:

from math import *

def sin_degrees(x):
    return sin(degrees(x))

Google Python Style Guide & relative imports

This is a reference to the Python 2 behavior (deprecated since 2.6) of implicit relative imports: allowing import bar in a module in a package foo to refer to the module foo.bar. Consider a directory on sys.path that looks like

…
|-- client.py
`-- pkg
    |-- __init__.py
    |-- mod.py
    `-- script.py

with files having the following contents:

`client.py`

print "client..."
from pkg import mod,script
print "client!"

`pkg/init.py`

print "pkg"

`pkg/mod.py`

print "mod: %r"%__name__

`pkg/script.py`

print "script:",__name__,__package__

if __name__=='__main__':
  import mod,client
  print "script!"

In this setup mod can easily be imported twice:

$ PYTHONPATH=… python …/pkg/script.py
script: __main__ None
mod: 'mod'
client...
pkg
mod: 'pkg.mod'
script: pkg.script None
client!
script!

In an attempt to reduce configuration overhead, Python adds the directory pkg to sys.path, effectively presuming that script.py is a top-level module script. Unfortunately, that means that import mod creates a top-level module named mod, and the explicit import of pkg.mod later causes another copy of it to exist with its full name (just after importing pkg itself).

It was recognized that this poses a problem, and later -m was adjusted to tell the module being executed about the package in which it was found so that relative imports (implicit or explicit) work properly:

$ PYTHONPATH=… python -m pkg.script
pkg
script: __main__ pkg
mod: 'pkg.mod'
client...
script: pkg.script None
client!
script!

Note that pkg is now imported first (by -m itself!), that script now has a __package__ attribute immediately afterwards, and that mod is imported just once. Of course, script itself is (still) loaded twice, since its name gets replaced with __main__ the first time so that from pkg import script finds it under a different name.

The moral is that implementing "a module can also be a script" in terms of __name__=='__main__' is fundamentally broken (and replacements have been rejected): a module already has a __name__, and creating a separate module object to be the entry point so that its __name__ can differ is as absurd as copying the Java class (and all its static data) that provides main. Making a module that no one ever imports works, but is oxymoronic (and breaks code inspection that imports all members of a package).

Relative Imports in Python 3