Importing from a relative path in Python
EDIT Nov 2014 (3 years later):
Python 2.6 and 3.x supports proper relative imports, where you can avoid doing anything hacky. With this method, you know you are getting a relative import rather than an absolute import. The '..' means, go to the directory above me:
from ..Common import Common
As a caveat, this will only work if you run your python as a module, from outside of the package. For example:
python -m Proj
Original hacky way
This method is still commonly used in some situations, where you aren't actually ever 'installing' your package. For example, it's popular with Django users.
You can add Common/ to your sys.path (the list of paths python looks at to import things):
import sys, os
sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'Common'))
import Common
os.path.dirname(__file__)
just gives you the directory that your current python file is in, and then we navigate to 'Common/' the directory and import 'Common' the module.
Import a module from a relative path
Assuming that both your directories are real Python packages (do have the __init__.py
file inside them), here is a safe solution for inclusion of modules relatively to the location of the script.
I assume that you want to do this, because you need to include a set of modules with your script. I use this in production in several products and works in many special scenarios like: scripts called from another directory or executed with python execute instead of opening a new interpreter.
import os, sys, inspect
# realpath() will make your script run, even if you symlink it :)
cmd_folder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile( inspect.currentframe() ))[0]))
if cmd_folder not in sys.path:
sys.path.insert(0, cmd_folder)
# Use this if you want to include modules from a subfolder
cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"subfolder")))
if cmd_subfolder not in sys.path:
sys.path.insert(0, cmd_subfolder)
# Info:
# cmd_folder = os.path.dirname(os.path.abspath(__file__)) # DO NOT USE __file__ !!!
# __file__ fails if the script is called in different ways on Windows.
# __file__ fails if someone does os.chdir() before.
# sys.argv[0] also fails, because it doesn't not always contains the path.
As a bonus, this approach does let you force Python to use your module instead of the ones installed on the system.
Warning! I don't really know what is happening when current module is inside an egg
file. It probably fails too.
Relative paths in Python
In the file that has the script, you want to do something like this:
import os
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'relative/path/to/file/you/want')
This will give you the absolute path to the file you're looking for. Note that if you're using setuptools, you should probably use its package resources API instead.
UPDATE: I'm responding to a comment here so I can paste a code sample. :-)
Am I correct in thinking that
__file__
is not always available (e.g. when you run the file directly rather than importing it)?
I'm assuming you mean the __main__
script when you mention running the file directly. If so, that doesn't appear to be the case on my system (python 2.5.1 on OS X 10.5.7):
#foo.py
import os
print os.getcwd()
print __file__
#in the interactive interpreter
>>> import foo
/Users/jason
foo.py
#and finally, at the shell:
~ % python foo.py
/Users/jason
foo.py
However, I do know that there are some quirks with __file__
on C extensions. For example, I can do this on my Mac:
>>> import collections #note that collections is a C extension in Python 2.5
>>> collections.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-
dynload/collections.so'
However, this raises an exception on my Windows machine.
Python imports relative path
You will need an __init__.py
in the mydir directory (and it can be empty), then as long as dir is in the sys path, assuming your MySampleClass is in myfile.py and myfile.py is in mydir
from mydir.myfile import MySampleClass
If you want to import top level functions from a file called util.py that reside in subdirA into myfile.py (where your class is), then an __init__.py
must be in subdirA and then in myfile.py
from subdirA.util import somefunc, someotherfunc
The same is true of the sys path, that is, you must either start from 'dir' or add it. Everything is imported from the top level of the package (usually your project folder).
However, for module testing, where you might run a function from util in the interpreter, if you start from subdirA, you will need to add dir to the sys path, so your imports can resolve.
>>> import sys
>>> sys.path.append('../dir')
but this is a hack and would be preferable to only use from the interactive interpreter when you are testing. You can also add 'dir' to your site packages in a pth file.
To use relative imports, you would need a deeper nested folder, like subdirA/subdirofA, then in subdirofA, you could use . to back out (like from .subdirB
). Really, for me, relative imports are somewhat difficult to see the utility. It's better for me to use direct imports relative to the project directory, but I could see them being useful if you wanted to nest a naive sub package, but again, still better to be explicit than implicit if possible.
also see this
Update for Python 3's namespace packages. An __init__.py
file is no longer required. Some libraries may still reference the file, so it may still be needed for compatibility. For example,
(test-env) machine:~ user$ mkdir new_package
(test-env) machine:~ user$ python
>>> import new_package
>>> new_package.__file__
>>> type(new_package.__file__)
<class 'NoneType'>
>>>
(test-env) machine:~ user$ touch new_package/__init__.py
(test-env) machine:~ user$ python
>>> import new_package
>>> new_package.__file__
'/Users/user/new_package/__init__.py'
>>> type(new_package.__file__)
<class 'str'>
>>>
Reading file using relative path in python project
Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__
special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]
How to do relative imports in Python?
Everyone seems to want to tell you what you should be doing rather than just answering the question.
The problem is that you're running the module as '__main__' by passing the mod1.py as an argument to the interpreter.
From PEP 328:
Relative imports use a module's __name__ attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to '__main__') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.
In Python 2.6, they're adding the ability to reference modules relative to the main module. PEP 366 describes the change.
Update: According to Nick Coghlan, the recommended alternative is to run the module inside the package using the -m switch.
Relative imports for the billionth time
Script vs. Module
Here's an explanation. The short version is that there is a big difference between directly running a Python file, and importing that file from somewhere else. Just knowing what directory a file is in does not determine what package Python thinks it is in. That depends, additionally, on how you load the file into Python (by running or by importing).
There are two ways to load a Python file: as the top-level script, or as a
module. A file is loaded as the top-level script if you execute it directly, for instance by typing python myfile.py
on the command line. It is loaded as a module when an import
statement is encountered inside some other file. There can only be one top-level script at a time; the top-level script is the Python file you ran to start things off.
Naming
When a file is loaded, it is given a name (which is stored in its __name__
attribute).
- If it was loaded as the top-level script, its name is
__main__
. - If it was loaded as a module, its name is [ the filename, preceded by the names of any packages/subpackages of which it is a part, separated by dots ], for example,
package.subpackage1.moduleX
.
But be aware, if you load moduleX
as a module from shell command line using something like python -m package.subpackage1.moduleX
, the __name__
will still be __main__
.
So for instance in your example:
package/
__init__.py
subpackage1/
__init__.py
moduleX.py
moduleA.py
if you imported moduleX
(note: imported, not directly executed), its name would be package.subpackage1.moduleX
. If you imported moduleA
, its name would be package.moduleA
. However, if you directly run moduleX
from the command line, its name will instead be __main__
, and if you directly run moduleA
from the command line, its name will be __main__
. When a module is run as the top-level script, it loses its normal name and its name is instead __main__
.
Accessing a module NOT through its containing package
There is an additional wrinkle: the module's name depends on whether it was imported "directly" from the directory it is in or imported via a package. This only makes a difference if you run Python in a directory, and try to import a file in that same directory (or a subdirectory of it). For instance, if you start the Python interpreter in the directory package/subpackage1
and then do import moduleX
, the name of moduleX
will just be moduleX
, and not package.subpackage1.moduleX
. This is because Python adds the current directory to its search path when the interpreter is entered interactively; if it finds the to-be-imported module in the current directory, it will not know that that directory is part of a package, and the package information will not become part of the module's name.
A special case is if you run the interpreter interactively (e.g., just type python
and start entering Python code on the fly). In this case, the name of that interactive session is __main__
.
Now here is the crucial thing for your error message: if a module's name has no dots, it is not considered to be part of a package. It doesn't matter where the file actually is on disk. All that matters is what its name is, and its name depends on how you loaded it.
Now look at the quote you included in your question:
Relative imports use a module's name attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to 'main') then relative imports are resolved as if the module were a top-level module, regardless of where the module is actually located on the file system.
Relative imports...
Relative imports use the module's name to determine where it is in a package. When you use a relative import like from .. import foo
, the dots indicate to step up some number of levels in the package hierarchy. For instance, if your current module's name is package.subpackage1.moduleX
, then ..moduleA
would mean package.moduleA
. For a from .. import
to work, the module's name must have at least as many dots as there are in the import
statement.
... are only relative in a package
However, if your module's name is __main__
, it is not considered to be in a package. Its name has no dots, and therefore you cannot use from .. import
statements inside it. If you try to do so, you will get the "relative-import in non-package" error.
Scripts can't import relative
What you probably did is you tried to run moduleX
or the like from the command line. When you did this, its name was set to __main__
, which means that relative imports within it will fail, because its name does not reveal that it is in a package. Note that this will also happen if you run Python from the same directory where a module is, and then try to import that module, because, as described above, Python will find the module in the current directory "too early" without realizing it is part of a package.
Also remember that when you run the interactive interpreter, the "name" of that interactive session is always __main__
. Thus you cannot do relative imports directly from an interactive session. Relative imports are only for use within module files.
Two solutions:
If you really do want to run
moduleX
directly, but you still want it to be considered part of a package, you can dopython -m package.subpackage1.moduleX
. The-m
tells Python to load it as a module, not as the top-level script.Or perhaps you don't actually want to run
moduleX
, you just want to run some other script, saymyfile.py
, that uses functions insidemoduleX
. If that is the case, putmyfile.py
somewhere else – not inside thepackage
directory – and run it. If insidemyfile.py
you do things likefrom package.moduleA import spam
, it will work fine.
Notes
For either of these solutions, the package directory (
package
in your example) must be accessible from the Python module search path (sys.path
). If it is not, you will not be able to use anything in the package reliably at all.Since Python 2.6, the module's "name" for package-resolution purposes is determined not just by its
__name__
attributes but also by the__package__
attribute. That's why I'm avoiding using the explicit symbol__name__
to refer to the module's "name". Since Python 2.6 a module's "name" is effectively__package__ + '.' + __name__
, or just__name__
if__package__
isNone
.)
Related Topics
Writing to Existing Workbook Using Xlwt
Where to Put Freeze_Support() in a Python Script
Why Doesn't Os.Path.Join() Work in This Case
What's the Difference Between Str.Isdigit, Isnumeric and Isdecimal in Python
How to Create a Read-Only Class Property in Python
Failed to Get Convolution Algorithm. This Is Probably Because Cudnn Failed to Initialize,
Calculate Weighted Average Using a Pandas/Dataframe
Pycharm Current Working Directory
Count Consecutive Occurences of Values Varying in Length in a Numpy Array
Appending to an Empty Dataframe in Pandas
Regular Expressions: Search in List