What Is the Purpose of "Pip Install --User ..."

What is the purpose of pip install --user ... ?

pip defaults to installing Python packages to a system directory (such as /usr/local/lib/python3.4). This requires root access.

--user makes pip install packages in your home directory instead, which doesn't require any special privileges.

Why my pip installations requires python -m install --user?

Let's break the two statements you added to your first command:
python -m install --user django

python -m: Allows modules to be located using the Python module namespace for execution as scripts. The motivating examples were standard library modules such as pdb and profile. See PEP 338
--user : By default pip installs Python packages to system directories which requires root privileges, to avoid using sudo pip install (which is not recommended by the way) use this flag to make pip install packages in your home directory instead, which doesn't require any special privileges.

As a side note, if you have multiple versions of Python installed, keeping track of which Python version version pip is bound to can be a PITA, hence python -m in this case you're sure that it's the pip bound to the Python called which will be executed.

What is the use case for `pip install -e`?

I find pip install -e extremely useful when simultaneously developing a product and a dependency, which I do a lot.

Example:

You build websites using Django for numerous clients, and have also developed an in-house Django app called locations which you reuse across many projects, so you make it available on pip and version it.

When you work on a project, you install the requirements as usual, which installs locations into site packages.

But you soon discover that locations could do with some improvements.

So you grab a copy of the locations repository and start making changes. Of course, you need to test these changes in the context of a Django project.

Simply go into your project and type:

pip install -e /path/to/locations/repo

This will overwrite the directory in site-packages with a symbolic link to the locations repository, meaning any changes to code in there will automatically be reflected - just reload the page (so long as you're using the development server).

The symbolic link looks at the current files in the directory, meaning you can switch branches to see changes or try different things etc...

The alternative would be to create a new version, push it to pip, and hope you've not forgotten anything. If you have many such in-house apps, this quickly becomes untenable.

Pip install python package for current user

Use pip's --user option:

pip install --user package

According to pip's documentation:

--user
Install to the Python user install directory for your platform. Typically ~/.local/, or %APPDATA%Python on Windows. (See the Python documentation for site.USER_BASE for full details.)

As a pip install user, am I supposed to have wheel installed?

This was a pip bug, and the solution is to upgrade the pip. With the newest version things look fine:

(venv) paulius@xps:~/Documents/wheeltest$ pip install databricks-cli
Collecting databricks-cli
  Using cached databricks-cli-0.14.3.tar.gz (54 kB)
Collecting click>=6.7
  Using cached click-8.0.1-py3-none-any.whl (97 kB)
Collecting requests>=2.17.3
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six>=1.10.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tabulate>=0.7.7
  Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.1-py3-none-any.whl (35 kB)
Using legacy 'setup.py install' for databricks-cli, since package 'wheel' is not installed.
Installing collected packages: urllib3, idna, charset-normalizer, certifi, tabulate, six, requests, click, databricks-cli
    Running setup.py install for databricks-cli ... done
Successfully installed certifi-2021.5.30 charset-normalizer-2.0.1 click-8.0.1 databricks-cli-0.14.3 idna-3.2 requests-2.26.0 six-1.16.0 tabulate-0.8.9 urllib3-1.26.6

Note the Using legacy 'setup.py install' ... line.

This is a related issue in the pip github https://github.com/pypa/pip/issues/8302. Not exactly that, but there is an explanation in the comments on what's the wheel building logic supposed to by.

When would the -e, --editable option be useful with pip install?

As the man page says it:

-e,--editable <path/url>
     Install a project in editable mode (i.e.  setuptools "develop mode") from a local project path or a VCS url.

So you would use this when trying to install a package locally, most often in the case when you are developing it on your system. It will just link the package to the original location, basically meaning any changes to the original package would reflect directly in your environment.

Some nuggets around the same here and here.

An example run can be:

pip install -e .

pip install -e ~/ultimate-utils/ultimate-utils-proj-src/

note the second is the full path to where the setup.py would be at.

What is the use of pip's --user option on Windows?

It is indeed confusing how the Windows Python installer handles install location with default settings (which you most probably used).

According to the documentation, when you run the installer and just click "Install Now":

You will not need to be an administrator (unless a system update for the C Runtime Library is required or you install the Python Launcher for Windows for all users)

Python will be installed into your user directory

The Python Launcher for Windows will be installed according to the option at the bottom of the first page.

Now, the option for the Python Launcher is also selected by default. This means, if your user account is in the "Administrator" group (which it typically is), Python (python.exe) will be installed in your %LocalAppData% directory (just as you have observed). However, the installer will still present you with a UAC prompt to confirm you have admin privileges, as it also wants to install the Python Launcher (py.exe). You end up with a local, "just for me" Python installation in your user directory, though for some intangible reason the Launcher is installed "for all users".

If you really want to have a system-wide Python installation, you need to select "Customize installation" on the first screen of the installer, then click "Next", and check "Install for all users" — which is not checked otherwise. The install location will then default to your %ProgramFiles% directory, usually C:\Program Files.

When you do user-installs with pip, it will put the packages in your %AppData% directory, which is AppData\Roaming in your user profile (as you have also observed). This is so that when you have a "roaming" account on a domain network, your personally installed packages follow you around, no matter from which computer on the network you log in. Obviously, that computer would have to have a system-wide Python installation "for all users" — of the Python interpreter, not the nearly irrelevant Python Launcher. This is where the default behavior, described above, makes absolutely zero sense, as you wouldn't be able to run the Python interpreter installed locally in some other user's profile when you log on to their computer.

On top of that, if you actually do use your profile to "roam" the domain network, all those --user packages, which, more likely than not, comprise thousands of files, will slow down the log-in process: every one of those files has to be sync'ed between domain storage and the local computer.

Bottom line: If you want to set this up properly, customize your installation to make sure it's installed in some directory that is in fact accessible to all users. Personally, I like to put it in C:\programs\Python, as I can then pip install anything for everyone and don't even need an elevated prompt — which one would for writing to C:\Program Files. Then again, requiring an elevated prompt may be advisable, depending on the circumstances.

Permission denied with pip install --user -e /home/me/package/

It's a bug. The issue is being tracked at https://github.com/pypa/pip/issues/7953 .

The workaround for now is to modify your setup.py() to contain this:

import site
import sys
site.ENABLE_USER_SITE = "--user" in sys.argv[1:]

Or to use something like this on the command line, if you use setup.cfg or pyproject.toml:

$ python3 -c 'import setuptools, site, sys; site.ENABLE_USER_SITE = 1; sys.argv[1:] = ["develop", "--user"]; setuptools.setup()'

What Is the Purpose of "Pip Install --User ..."