Python: Interplay Between Lib/Site-Packages/Site.Py and Lib/Site.Py

python: Interplay between lib/site-packages/site.py and lib/site.py

The lib/python2.7/site-packages/site.py file is not normally loaded. That's because it is lib/python2.7/site.py's job to add the site-packages paths to sys.path to begin with, and the site.py in site-packages is simply not visible. If you have a site.py in site-packages then that is an error, there should be no such file there.

What happens in an unpatched Python without is:

• Python starts, with a limited sys.path. site-packages is not part of this list, unless you set a PYTHONPATH variable that includes it anyway.
• Python imports site.py, it'll import the one listed first on sys.path.
• lib/python2.7/site.py is found and loaded
• site.py adds site-packages to sys.path

That's it, no further site.py modules are loaded. Even if you tried, it'd find the module that was already imported; sys.modules['site'] exists and holds objects loaded from lib/python2.7/site.py.

Your installation, however, has an older setuptools installed, and it includes a special version of site.py, which the easy_install command will install into site-packages if not yet present. It'll load the original site.py by explicitly scanning the original sys.path with any PYTHONPATH-supplied paths ignored and loading the original site.py module manually using the imp.find_module() and imp.load_module() low-level functions, thus bypassing the normal module cache.

Its intent was to alter the sys.path order to give PYTHONPATH-listed .pth files a higher precedence, see the original commit adding the patch:

Note: this version includes a hacked 'site.py' to support processing
.pth files in directories that come before site-packages on sys.path.

The patch has been removed entirely from more recent setuptools releases, as early as 2006 in the original setuptools.

So, either your Linux distribution has been set up to add lib/python2.7/site-packages to your PYTHONPATH or your shell has this set up for you, or your Python has been patched to include it, and you have the old setuptools 'patch' in your site-packages.

It is entirely safe to remove that file.

Python's 'site.py' gone after Yosemite upgrade. Is that okay?

site.py is still used. You are just not looking in the right location:

>>> import site
>>> print site.__file__
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site.pyc

The /Extras structure appears to consist entirely of non-standard-library packages, e.g. packages that Apple installs for their own uses that are not included with standard Python.

If there was a site.py file there in previous OS X versions it was in all likelihood one installed by setuptools; with 10.10 comes setuptools 1.1.6, which has long since got rid of the hack embodied in that file.

Pyinstaller failed to import the site module

Something is amiss with your Python installation. The "site-specific configuration hook" site.py is normally found in the Lib folder just underneath the Python install directory. But according to your error log, PyInstaller finds it in Lib/site-packages.

See also this answer from 2014:

If you have a site.py in site-packages then that is an error, there should be no such file there.

It's referring to Python 2.7, but explains well what site.py does and still applies to newer Python versions, such as Python 3.9 here.

Furthermore, the code line

f = io.TextIOWrapper(io.open_code(fullname), encoding="locale")

in your custom site.py is wrong. There is no (standard) encoding with the name 'locale'. Which is why it raises that LookupError. That same line of code in the site.py included with Python 3.9 does not use the optional encoding argument.

Fatal Python error: init_import_size: Failed to import the site module in Anaconda Prompt

Are you running another python3 set up in your machine ? Because this error occurs when more than one python environment is there. The best solution is to uninstall other python versions set up and do clean installation. Other solutions will be.
a) Check your .bashrc file (if using Ubuntu) and remove the PATH variables to other python versions and only keep the PATH variable for Anaconda.

b) Run the python scripts specific python binary.

/home/username/Programs/anaconda3/bin/python3 file.py

I hope, these solutions will work

prevent python from loading a pth file

I finally managed to solve this - my site-packages library had an easy_install.pth file containing the faulty numpy for some reason, and not the x.pth file on /usr/local/lib/site-packages.

After the major amount of time I spent on this, I'll share some of the stuff I've learned if someone else ever gets here:

• If you have a locally installed python it will not search in '/usr/local/lib' by default. If you want to know where it searches, run:

  import site
  print site.PREFIXES
  

  python searches for the paths here + lib/python2.X/site-packages. It's described here

• According to these docs, you can in fact play around with your sys.path. If you add usercustomize module to your local sitepackages (import site; site.getusersitepackages()), it will be ran. However, and this took me time to realize - it happens after python processes the pth files located at your site-packages, and before he processes them AGAIN. If I add a print statement to the method that does this (lib/site.py addsitedir), this is what it will print:

  /home/user/.local/lib/python2.7/site-packages
  /home/user/py/lib/python2.7/site-packages
  #This is where your code runs
  /home/user/py/lib/python2.7/site-packages/numpy-1.9.0-py2.7-linux-x86_64.egg
  /home/user/Develop/Python/myproject
  /home/user/lmfit-0.7.2
  /home/user/py/lib/python2.7/site-packages #NOTE: this runs a second time
  

• Once I had a pth file that I missed in site-packages, I couldn't fix it with a usercustomize, since that pth file got a chance to run one more time afterwards!

Listing installed python site-packages?

Check out yolk.

Yolk is a Python command-line tool and library for obtaining information about packages installed by setuptools, easy_install and distutils (Python 2.5) and for querying PyPI (Python Package Index a.k.a. The Cheese Shop).

Can't load Python modules installed via pip from site-packages directory

/usr/bin/python is the executable for the python that comes with OS X. /usr/local/lib is a location for user-installed programs only, possibly from Python.org or Homebrew. So you're mixing different Python installs, and changing the python path is only a partial workaround for different packages being installed for different installations.

In order to make sure you use the pip associated with a particular python, you can run python -m pip install <pkg>, or go look at what the pip on your path is, or is symlinked to.

---

Related Topics