Python: Best Way to Add to Sys.Path Relative to the Current Running Script

Python: Best way to add to sys.path relative to the current running script

If you don't want to edit each file

• Install you library like a normal python libray
  
  or
• Set PYTHONPATH to your lib

or if you are willing to add a single line to each file, add a import statement at top e.g.

import import_my_lib

keep import_my_lib.py in bin and import_my_lib can correctly set the python path to whatever lib you want

Relative paths in Python

In the file that has the script, you want to do something like this:

import os
dirname = os.path.dirname(__file__)
filename = os.path.join(dirname, 'relative/path/to/file/you/want')

This will give you the absolute path to the file you're looking for. Note that if you're using setuptools, you should probably use its package resources API instead.

UPDATE: I'm responding to a comment here so I can paste a code sample. :-)

Am I correct in thinking that __file__ is not always available (e.g. when you run the file directly rather than importing it)?

I'm assuming you mean the __main__ script when you mention running the file directly. If so, that doesn't appear to be the case on my system (python 2.5.1 on OS X 10.5.7):

#foo.py
import os
print os.getcwd()
print __file__

#in the interactive interpreter
>>> import foo
/Users/jason
foo.py

#and finally, at the shell:
~ % python foo.py
/Users/jason
foo.py

However, I do know that there are some quirks with __file__ on C extensions. For example, I can do this on my Mac:

>>> import collections #note that collections is a C extension in Python 2.5
>>> collections.__file__
'/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/lib-
dynload/collections.so'

However, this raises an exception on my Windows machine.

sys.path.append modules relative to path of script being run

By doing some research seems this could be a possible answer:

# Import modules
import subprocess, sys, os
script_path = os.path.dirname(__file__)
myutils_path = os.path.join(script_path, '../myutils')
sys.path.append(myutils_path)
import myutils

How to import files in python using sys.path.append?

You can create a path relative to a module by using a module's __file__ attribute. For example:

myfile = open(os.path.join(
    os.path.dirname(__file__),
    MY_FILE))

This should do what you want regardless of where you start your script.

Import a module from a relative path

Assuming that both your directories are real Python packages (do have the __init__.py file inside them), here is a safe solution for inclusion of modules relatively to the location of the script.

I assume that you want to do this, because you need to include a set of modules with your script. I use this in production in several products and works in many special scenarios like: scripts called from another directory or executed with python execute instead of opening a new interpreter.

 import os, sys, inspect
 # realpath() will make your script run, even if you symlink it :)
 cmd_folder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile( inspect.currentframe() ))[0]))
 if cmd_folder not in sys.path:
     sys.path.insert(0, cmd_folder)

 # Use this if you want to include modules from a subfolder
 cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"subfolder")))
 if cmd_subfolder not in sys.path:
     sys.path.insert(0, cmd_subfolder)

 # Info:
 # cmd_folder = os.path.dirname(os.path.abspath(__file__)) # DO NOT USE __file__ !!!
 # __file__ fails if the script is called in different ways on Windows.
 # __file__ fails if someone does os.chdir() before.
 # sys.argv[0] also fails, because it doesn't not always contains the path.

As a bonus, this approach does let you force Python to use your module instead of the ones installed on the system.

Warning! I don't really know what is happening when current module is inside an egg file. It probably fails too.

Reading file using relative path in python project

Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.

But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:

from pathlib import Path

path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
    test = list(csv.reader(f))

This requires python 3.4+ (for the pathlib module).

If you still need to support older versions, you can get the same result with:

import csv
import os.path

my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
    test = list(csv.reader(f))

[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]

Permanently adding a file path to sys.path in Python

There are a few ways. One of the simplest is to create a my-paths.pth file (as described here). This is just a file with the extension .pth that you put into your system site-packages directory. On each line of the file you put one directory name, so you can put a line in there with /path/to/the/ and it will add that directory to the path.

You could also use the PYTHONPATH environment variable, which is like the system PATH variable but contains directories that will be added to sys.path. See the documentation.

Note that no matter what you do, sys.path contains directories not files. You can't "add a file to sys.path". You always add its directory and then you can import the file.

Get parent of current directory from Python script

Using os.path

To get the parent directory of the directory containing the script (regardless of the current working directory), you'll need to use __file__.

Inside the script use os.path.abspath(__file__) to obtain the absolute path of the script, and call os.path.dirname twice:

from os.path import dirname, abspath
d = dirname(dirname(abspath(__file__))) # /home/kristina/desire-directory

Basically, you can walk up the directory tree by calling os.path.dirname as many times as needed. Example:

In [4]: from os.path import dirname

In [5]: dirname('/home/kristina/desire-directory/scripts/script.py')
Out[5]: '/home/kristina/desire-directory/scripts'

In [6]: dirname(dirname('/home/kristina/desire-directory/scripts/script.py'))
Out[6]: '/home/kristina/desire-directory'

If you want to get the parent directory of the current working directory, use os.getcwd:

import os
d = os.path.dirname(os.getcwd())

Using pathlib

You could also use the pathlib module (available in Python 3.4 or newer).

Each pathlib.Path instance have the parent attribute referring to the parent directory, as well as the parents attribute, which is a list of ancestors of the path. Path.resolve may be used to obtain the absolute path. It also resolves all symlinks, but you may use Path.absolute instead if that isn't a desired behaviour.

Path(__file__) and Path() represent the script path and the current working directory respectively, therefore in order to get the parent directory of the script directory (regardless of the current working directory) you would use

from pathlib import Path
# `path.parents[1]` is the same as `path.parent.parent`
d = Path(__file__).resolve().parents[1] # Path('/home/kristina/desire-directory')

and to get the parent directory of the current working directory

from pathlib import Path
d = Path().resolve().parent

Note that d is a Path instance, which isn't always handy. You can convert it to str easily when you need it:

In [15]: str(d)
Out[15]: '/home/kristina/desire-directory'

---

Related Topics