Does 'Anaconda' Create a Separate Pythonpath Variable for Each New Environment

Does `anaconda` create a separate PYTHONPATH variable for each new environment?

No, the only thing that needs to be modified for an Anaconda environment is the PATH (so that it gets the right Python from the environment bin/ directory, or Scripts\ on Windows).

The way Anaconda environments work is that they hard link everything that is installed into the environment. For all intents and purposes, this means that each environment is a completely separate installation of Python and all the packages. By using hard links, this is done efficiently. Thus, there's no need to mess with PYTHONPATH because the Python binary in the environment already searches the site-packages in the environment, and the lib of the environment, and so on.

PYTHONPATH environment variable - default vs anaconda

If you install Python directly from here, your PYTHONPATH is going to look something like /usr/local/bin/python3. If you install Anaconda instead, then Python will come with it, and its PYTHONPATH will look like /Users/<name>/anaconda3/bin/python.

There is practically no difference between these two Python installations per se, but using Anaconda makes installations easier to manage. Especially if you will use other tools available in Anaconda environment.

Unexpected Python paths in Conda environment

This is expected behavior (see PEP 370) and partially why Anaconda recommended against user-level package installations.

The site module is responsible for setting the sys.path when Python is initializing. The code in site.py specifically appends the user site prior to appending the prefix site, which is what leads to this prioritization. The motivation according to PEP 370 is that users would have a Python installed at system-level, but want to prioritize packages they install at the user level, hence the user site should load prior to the prefix site.

Options

There are several options for avoiding the user-level site-packages from getting loaded.

1: Environment variable

The environment variable PYTHONNOUSERSITE will toggle loading of user-level site-packages. Namely,

PYTHONNOUSERSITE=1 python -c "import sys; print(sys.path)"

2: Python -s flag

Alternatively, the Python binary has an -s argument to specifically disable user-level site packages.

python -s -c "import sys; print(sys.path)"

3: Remove (and avoid future) user-level installs

The Conda recommendation is to avoid pip install --user altogether, which would be interpreted that one should remove the ~/.local/lib/python* folders from your system.

4: Automated Conda environment variable

Conda Forge package

The Conda Forge package conda-ecosystem-user-package-isolation will automatically set PYTHONNOUSERSITE=1 during environment activation.

If you would like all environments to have such isolation by default, then consider adding this to the create_default_packages configuration list:

conda config --add create_default_packages conda-ecosystem-user-package-isolation

Note that this package also sets R_LIBS_USER="-", isolating any R environments from user-level packages.

Alternative packages

If you want a more granular option, I have also created separate packages that set just the PYTHONNOUSERSITE=1 and PYTHONPATH="" environment variables, which can be installed with:

## set PYTHONNOUSERSITE=1
conda install merv::envvar-pythonnousersite-true

## clear PYTHONPATH
conda install merv::envvar-pythonpath-null

Anaconda: Permanently include external packages (like in PYTHONPATH)

I found two answers to my question in the Anaconda forum:

1.) Put the modules into into site-packages, i.e. the directory $HOME/path/to/anaconda/lib/pythonX.X/site-packages which is always on sys.path. This should also work by creating a symbolic link.

2.) Add a .pth file to the directory $HOME/path/to/anaconda/lib/pythonX.X/site-packages. This can be named anything (it just must end with .pth). A .pth file is just a newline-separated listing of the full path-names of directories that will be added to your path on Python startup.

Alternatively, if you only want to link to a particular conda environment then add the .pth file to ~/anaconda3/envs/{NAME_OF_ENVIRONMENT}/lib/pythonX.X/site-packages/

Both work straightforward and I went for the second option as it is more flexible.

*** UPDATE:

3.) Use conda develop i. e. conda-develop /path/to/module/ to add the module which creates a .pth file as described under option 2.).

4.) Create a setup.py in the folder of your package and install it using pip install -e /path/to/package which is the cleanest option from my point of view because you can also see all installations using pip list. Note that the option -e allows to edit the package code. See here for more information.

Thanks anyway!

Why has Anaconda added my default Python paths to the specific environment's path?

Check to see if you have the environment variables PYTHONPATH or PYTHONHOME set. conda info -a will also show you all the relevant environment variables that might cause this sort of thing to happen.

I have added conda and python path to the environment variable, but jupyter notebook is still not getting opened from cmd

Adding Python to the environment path is bad practice, see Anaconda FAQ. If you haven't installed Anaconda with it's default settings, you first need to:

Initialize your shells

conda init --all

After this you should have ../Anaconda3/condabin only in your path (more information via conda init --help).

But before you can run Jupyter, you also need to activate Anaconda:

C:\> conda activate
(base) C:\> jupyter notebook

The activation will add the following folders of the conda base environment to your PATH:

\Anaconda3;
\Anaconda3\Library\mingw-w64\bin;
\Anaconda3\Library\usr\bin;
\Anaconda3\Library\bin;
\Anaconda3\Scripts;
\Anaconda3\bin;

The python.exe resides in Anaconda3, jupyter.exe in Anaconda3\Scripts, so it's not enough to just add the first folder to your Path. And it's especially important to have the libraries on your Path when you want to run C-based packages like numpy.

But the very point behind the conda activate mechanism is that it allows you to configure and run different environments with different versions of python and 3rd party packages that would otherwise conflict, see Managing environmnts.

On top of that you can even install Python from python.org next to your Anaconda distribution, since conda will make sure that they won't interfere.

Anaconda new environment gives me python SyntaxError 'yield' inside async function

[Moved from comments]

Somehow or other you're trying to import from your root install even within your activated env, and that root install (being Python 3.6) uses async + yield in its stdlib, which isn't supported in the 3.5 you want to use.

When using Anaconda, you shouldn't have either PYTHONPATH or PYTHONHOME set (and if there are other PYTHON* environment variables set, might as well clear them too!)

These cause problems because these variables are very powerful and the interpreter winds up obeying them. In the case of multiple environments and/or multiple Python distributions on the same system, it's best to leave them alone.

---

Related Topics