Expand Python Search Path to Other Source

Expand Python Search Path to Other Source

There are a few possible ways to do this:

• Set the environment variable PYTHONPATH to a colon-separated list of directories to search for imported modules.
• In your program, use sys.path.append('/path/to/search') to add the names of directories you want Python to search for imported modules. sys.path is just the list of directories Python searches every time it gets asked to import a module, and you can alter it as needed (although I wouldn't recommend removing any of the standard directories!). Any directories you put in the environment variable PYTHONPATH will be inserted into sys.path when Python starts up.
• Use site.addsitedir to add a directory to sys.path. The difference between this and just plain appending is that when you use addsitedir, it also looks for .pth files within that directory and uses them to possibly add additional directories to sys.path based on the contents of the files. See the documentation for more detail.

Which one of these you want to use depends on your situation. Remember that when you distribute your project to other users, they typically install it in such a manner that the Python code files will be automatically detected by Python's importer (i.e. packages are usually installed in the site-packages directory), so if you mess with sys.path in your code, that may be unnecessary and might even have adverse effects when that code runs on another computer. For development, I would venture a guess that setting PYTHONPATH is usually the best way to go.

However, when you're using something that just runs on your own computer (or when you have nonstandard setups, e.g. sometimes in web app frameworks), it's not entirely uncommon to do something like

import sys
from os.path import dirname
sys.path.append(dirname(__file__))

Permanently add a directory to PYTHONPATH?

You need to add your new directory to the environment variable PYTHONPATH, separated by a colon from previous contents thereof. In any form of Unix, you can do that in a startup script appropriate to whatever shell you're using (.profile or whatever, depending on your favorite shell) with a command which, again, depends on the shell in question; in Windows, you can do it through the system GUI for the purpose.

superuser.com may be a better place to ask further, i.e. for more details if you need specifics about how to enrich an environment variable in your chosen platform and shell, since it's not really a programming question per se.

How can I import a module dynamically given the full path?

For Python 3.5+ use (docs):

import importlib.util
import sys
spec = importlib.util.spec_from_file_location("module.name", "/path/to/file.py")
foo = importlib.util.module_from_spec(spec)
sys.modules["module.name"] = foo
spec.loader.exec_module(foo)
foo.MyClass()

For Python 3.3 and 3.4 use:

from importlib.machinery import SourceFileLoader

foo = SourceFileLoader("module.name", "/path/to/file.py").load_module()
foo.MyClass()

(Although this has been deprecated in Python 3.4.)

For Python 2 use:

import imp

foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()

There are equivalent convenience functions for compiled Python files and DLLs.

See also http://bugs.python.org/issue21436.

Visual Studio Code - How to add multiple paths to python path?

This worked for me:-

in your launch.json profile entry, specify a new entry called "env", and set PYTHONPATH yourself.

"configurations": [
    {
        "name": "Python",
        "type": "python",
        "stopOnEntry": false,
        "request": "launch",
        "pythonPath": "${config.python.pythonPath}",
        "program": "${file}",
        "cwd": "${workspaceRoot}",
        "debugOptions": [
            "WaitOnAbnormalExit",
            "WaitOnNormalExit",
            "RedirectOutput"
        ],
        "env": {
            "PYTHONPATH": "/path/a:path/b"
        }
    }
]

python search path for data files (non-modules) - text files (*txt)

There is no such option. Only the current working directory is searched if no path is specified in the filename.

Python: Best way to add to sys.path relative to the current running script

If you don't want to edit each file

• Install you library like a normal python libray
  
  or
• Set PYTHONPATH to your lib

or if you are willing to add a single line to each file, add a import statement at top e.g.

import import_my_lib

keep import_my_lib.py in bin and import_my_lib can correctly set the python path to whatever lib you want

Import a module from a relative path

Assuming that both your directories are real Python packages (do have the __init__.py file inside them), here is a safe solution for inclusion of modules relatively to the location of the script.

I assume that you want to do this, because you need to include a set of modules with your script. I use this in production in several products and works in many special scenarios like: scripts called from another directory or executed with python execute instead of opening a new interpreter.

 import os, sys, inspect
 # realpath() will make your script run, even if you symlink it :)
 cmd_folder = os.path.realpath(os.path.abspath(os.path.split(inspect.getfile( inspect.currentframe() ))[0]))
 if cmd_folder not in sys.path:
     sys.path.insert(0, cmd_folder)

 # Use this if you want to include modules from a subfolder
 cmd_subfolder = os.path.realpath(os.path.abspath(os.path.join(os.path.split(inspect.getfile( inspect.currentframe() ))[0],"subfolder")))
 if cmd_subfolder not in sys.path:
     sys.path.insert(0, cmd_subfolder)

 # Info:
 # cmd_folder = os.path.dirname(os.path.abspath(__file__)) # DO NOT USE __file__ !!!
 # __file__ fails if the script is called in different ways on Windows.
 # __file__ fails if someone does os.chdir() before.
 # sys.argv[0] also fails, because it doesn't not always contains the path.

As a bonus, this approach does let you force Python to use your module instead of the ones installed on the system.

Warning! I don't really know what is happening when current module is inside an egg file. It probably fails too.

Customize module search path (PYTHONPATH) via pipenv

I'm not sure there's a perfect solution for this, but in the interest of being explicit rather than implicit (PEP 20), I've decided to check in a file that needs to be sourced before running any script. This is one extra manual step but you can put this in a Makefile for instance.

env.sh

export PYTHONPATH=${PYTHONPATH}:${PWD}

Makefile

bar:
    source env.sh && pipenv run python scripts/bar.py
.PHONY: migrate

The solution is a bit similar to the approach Go takes with its GOPATH.

I think the other solutions are not as good:

• pipenv aims to solve dependencies, I could be wrong but I did not find anything related to the problem of the PYTHONPATH.
• Linking folders won't scale really well if you start having other scripts folder.

---

Related Topics