When to Use Sys.Path.Append and When Modifying %Pythonpath% Is Enough

When to use sys.path.append and when modifying %PYTHONPATH% is enough

Perhaps you're not running the scheduled task under the right credentials (log-in name). When you define environment variables in System Properties dialog, they can be either User-level or System-level. If you defined PYTHONPATH as User-level then your scheduled task must run as that user for it to be set properly. I believe making it System-level would mean it would apply to all users unless they have their own value defined.

Below is a screenshot showing where one sets environment variables. It's similar in both Windows XP and Windows 7. The top half of the right-hand dialog box shows the current User-level settings, and the bottom half lists all the System-level ones.

If PYTHONPATH appears in the list of names in the upper User-level group, you can effectively move it to the other lower one by first deleting and then adding one of the same name plus associated value to the lower System-level set. To save a little typing, you can Edit the user-level variable before you Delete it to be given a chance to first copy its current value, then Cancel-out of the operation. That way, when you make the New System-level copy you'll be able to simply paste the copied value into it.

PYTHONPATH vs. sys.path

If the only reason to modify the path is for developers working from their working tree, then you should use an installation tool to set up your environment for you. virtualenv is very popular, and if you are using setuptools, you can simply run setup.py develop to semi-install the working tree in your current Python installation.

Why do I need to include sys.path.append to import a module with Python 3.6 and my colleges doesn't need?

Just set the PYTHONPATH environment variable to (the full path to) Myproject. That tells Python where to find modules to import, and it works regardless of which directory you run the script from. This avoids the need to modify sys.path.

Effect of using sys.path.insert(0, path) and sys.path(append) when loading modules

Because python checks in the directories in sequential order starting at the first directory in sys.path list, till it find the .py file it was looking for.

Ideally, the current directory or the directory of the script is the first always the first element in the list, unless you modify it, like you did. From documentation -

As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.

So, most probably, you had a .py file with the same name as the module you were trying to import from, in the current directory (where the script was being run from).

Also, a thing to note about ImportErrors , lets say the import error says -
ImportError: No module named main - it doesn't mean the main.py is overwritten, no if that was overwritten we would not be having issues trying to read it. Its some module above this that got overwritten with a .py or some other file.

Example -

My directory structure looks like -

 - test
    - shared
         - __init__.py
         - phtest.py
  - testmain.py

Now From testmain.py , I call from shared import phtest , it works fine.

Now lets say I introduce a shared.py in test directory` , example -

 - test
    - shared
         - __init__.py
         - phtest.py
  - testmain.py 
  - shared.py

Now when I try to do from shared import phtest from testmain.py , I will get the error -

ImportError: cannot import name 'phtest'

As you can see above, the file that is causing the issue is shared.py , not phtest.py .

change the priority of pythonpath

Yes, you can import sys and manipulate sys.path as an ordinary list at runtime. You can rearrange what's there, or just insert at the beginning (sys.path.insert(0, 'path')). Do this before your import statement. If this will cause problems elsewhere, put it back after your import statement.

Note, this is fairly hacky. But it sounds like you might have a case for it, although I have not looked at these specific tools together.

Edit: this is more relevant if you want to control the python path at the level of individual imports within the course of one execution of python. If you want to control the path at the level of one full execution of Python, you can also set the python path on the command line for just that execution like this:

PYTHONPATH=/replacement/path/here python your_script.py

This is more verbose (unless you wrap it in a shell script or alias) than just calling python, but it lets you control the path one script at a time, wheres putting it into .bashrc/.bash_profile or similar changes it for your whole shell session.

Add a directory to Python sys.path so that it's included each time I use Python

Simply add this path to your PYTHONPATH environment variable. To do this, go to Control Panel / System / Advanced / Environment variable, and in the "User variables" sections, check if you already have PYTHONPATH. If yes, select it and click "Edit", if not, click "New" to add it.

Paths in PYTHONPATH should be separated with ";".

Permanently add a directory to PYTHONPATH?

You need to add your new directory to the environment variable PYTHONPATH, separated by a colon from previous contents thereof. In any form of Unix, you can do that in a startup script appropriate to whatever shell you're using (.profile or whatever, depending on your favorite shell) with a command which, again, depends on the shell in question; in Windows, you can do it through the system GUI for the purpose.

superuser.com may be a better place to ask further, i.e. for more details if you need specifics about how to enrich an environment variable in your chosen platform and shell, since it's not really a programming question per se.

Expand Python Search Path to Other Source

There are a few possible ways to do this:

• Set the environment variable PYTHONPATH to a colon-separated list of directories to search for imported modules.
• In your program, use sys.path.append('/path/to/search') to add the names of directories you want Python to search for imported modules. sys.path is just the list of directories Python searches every time it gets asked to import a module, and you can alter it as needed (although I wouldn't recommend removing any of the standard directories!). Any directories you put in the environment variable PYTHONPATH will be inserted into sys.path when Python starts up.
• Use site.addsitedir to add a directory to sys.path. The difference between this and just plain appending is that when you use addsitedir, it also looks for .pth files within that directory and uses them to possibly add additional directories to sys.path based on the contents of the files. See the documentation for more detail.

Which one of these you want to use depends on your situation. Remember that when you distribute your project to other users, they typically install it in such a manner that the Python code files will be automatically detected by Python's importer (i.e. packages are usually installed in the site-packages directory), so if you mess with sys.path in your code, that may be unnecessary and might even have adverse effects when that code runs on another computer. For development, I would venture a guess that setting PYTHONPATH is usually the best way to go.

However, when you're using something that just runs on your own computer (or when you have nonstandard setups, e.g. sometimes in web app frameworks), it's not entirely uncommon to do something like

import sys
from os.path import dirname
sys.path.append(dirname(__file__))

How to correctly set PYTHONPATH for Visual Studio Code

OP seemed to have asked about path syntax for the .env file and the vscode set up so that it finds and reads some custom module files. My problem was similar in that I wanted code to find my custom modules for import in a script. I did not want to put my custom modules in a folder inside my python environment. I also wanted to avoid setting one or more paths as PYTHONPATH for the User Variables in the Windows Environment Variables - but this will work if you want to do it.
I am working in vscode in Windows 10.

1) SYNTAX:

a) I found that the following path syntax works in the env file:

PYTHONPATH = C:/0APPS/PYTHON/_MODULES

My .py module files are in this folder.

b) # works for comments in the .env file.

2) VSCODE SET-UP: I found that the following works:

a) Like sunew said at #2 My setup: Use the Explorer in vscode to open at your selected project workspace folder. For me that is Q:\420 PYTHON

b) Give the env file a name, like vscode.env file and place it in that folder at the top level of the workspace.

c) Open vscode settings and search .env where under the Extensions > Python you will find "Python: env file". Edit the box to add your env file name just before .env.
e.g. ${workspaceFolder}/vscode.env

d) import custom_modulename now work for me - in the python interactive window and in a script.

Do I need to add my project directory to the system path in every script to import a function from another directory?

First of all, let me describe you the differences between a Python module & a Python package so that both of us are on the same page. ✌

---

A module is a single .py file (or files) that are imported under one import and used. ✔

import aModuleName
# Here 'aModuleName' is just a regular .py file.

Whereas, a package is a collection of modules in directories that give a package hierarchy. A package contains a distinct __init__.py file. ✔

from aPackageName import aModuleName
# Here 'aPackageName` is a folder with a `__init__.py` file
# and 'aModuleName', which is just a regular .py file.

---

Therefore, when we have a project directory named proj-dir of the following structure ⤵

proj-dir
--|--__init__.py
--package1
--|--__init__.py
--|--module1.py
--package2
--|--__init__.py
--|--module2.py

Notice that I've also added an empty __init__.py into the proj-dir itself which makes it a package too.

Now, if you want to import any python object from module2 of package2 into module1 of package1, then the import statement in the file module1.py would be

from package2.module2 import object2
# if you were to import the entire module2 then,
from package2 import module2

I hope this simple explanation clarifies your doubts on Python imports' mechanism and solves the problem. If not then do comment here. /h3>

---

Related Topics