Too Many Different Python Versions on My System and Causing Problems

Too many different Python versions on my system and causing problems

Why did it get messed up?

There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.

What problem does it cause?

I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.

How to avoid this?

I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.

• Unless you intend to, don't install the same thing twice via different
  ways. (If you intend to do it for python, you might want to check out virtualenv)
• Keep an eye on the path order in your PATH and consider if it's
  correct.
• When installing modules, be clear which python (or pip) is
  running and where the module is installed.

So, how did I solve my own case?

Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.

Dealing with multiple Python versions and PIP?

The current recommendation is to use python -m pip, where python is the version of Python you would like to use. This is the recommendation because it works across all versions of Python, and in all forms of virtualenv. For example:

# The system default python:
$ python -m pip install fish

# A virtualenv's python:
$ .env/bin/python -m pip install fish

# A specific version of python:
$ python-3.6 -m pip install fish

Previous answer, left for posterity:

Since version 0.8, Pip supports pip-{version}. You can use it the same as easy_install-{version}:

$ pip-2.5 install myfoopackage
$ pip-2.6 install otherpackage
$ pip-2.7 install mybarpackage

---

EDIT: pip changed its schema to use pipVERSION instead of pip-VERSION in version 1.5. You should use the following if you have pip >= 1.5:

$ pip2.6 install otherpackage
$ pip2.7 install mybarpackage

Check https://github.com/pypa/pip/pull/1053 for more details

---

References:

• https://github.com/pypa/pip/issues/200
• http://www.pip-installer.org/docs/pip/en/0.8.3/news.html#id4
  
  https://pip.pypa.io/en/stable/news/#v0-8 or
  
  https://web.archive.org/web/20140310013920/http://www.pip-installer.org:80/docs/pip/en/0.8.3/news.html#id4

MacOS: Multiple python versions installed and unmanageable. Cleanup wanted

Having multiple versions of python is not really a propblem per se.

What I recommend is :

# In $HOME/.bashrc or .zshrc
PATH=/usr/local/bin:$PATH

cd /usr/local/bin
ln -fs python3 python
# Once the first and this step done, when you type [python],
# you'll be using /usr/local/bin/python3

As the first line of your python scripts, put :

#!/usr/bin/env python

This way, you ensure your are always using the version /usr/local/bin/python3

I uninstalled some Python 2.7 essential files from my Mac, and my OS is now problematic. If I update the software will they be reinstalled?

Hmm, where did you uninstall from? Let's hope you didn't remove it from your System/Library! See this post here - the discussions/answers may provide some help: How to uninstall Python 2.7 on a Mac OS X 10.6.4?. And yes, @FishingCode could be right - an update may just solve the problem.

Steps you could take:

1) Try updating first!

2) If the update did not work AND your accidental deletions did not affect the system files, then try uninstalling any third-party Python files with:

rm -rf </Library/Frameworks/Python.framework/Versions/VERSION_NUMBER> (replace VERSION_NUMBER with the Python version you want to remove)

OR

If you installed with pip, pip uninstall <Python>

You can also remove the third-party Python 2.7 from the same path (check the same post I referenced above! The accepted answer details a good plan to follow!).

Then, reinstall the usual way (https://www.python.org).

3) Check with the Apple stores/Apple technician.

Overall Recommendations:
I do recommend you install a Python version >= 3! Python 2 support has been deprecated since last year and there are some considerable differences between Python 2 and Python 3.x. You can definitely install the latest version (3.8), too. Hope this helps :)

How to install multiple versions of Python in Windows?

Your questions depend a bit on "all the other software". For example, as @leiyang indicated, the answer will be different if you use conda vs just pip on vanilla CPython (the standard Windows Python).

I'm also going to assume you're actually on Windows, because on Linux I would recommend looking at pyenv. There is a pyenv-win, which may be worth looking into, but I don't use it myself because it doesn't play as nice if you also want (mini)conda environments.

1. (a) How do I install multiple Python versions?

Simply download the various installers and install them in sensible locations. E.g. "C:\Program Files\Python39" for Python 3.9, or some other location where you're allowed to install software.

Don't have Python add itself to the PATH though, since that'll only find the last version to do so and can really confuse things.

Also, you probably want to use virtual environments consistently, as this ties a specific project very clearly to a specific Python version, avoiding future confusion or problems.

1. (b) "3.8.1, 3.8.2 till 3.8.13" which should I pick?

Always pick the latest 3.x.y, so if there's a 3.8.13 for Windows, but no 3.8.14, pick that. Check if the version is actually available for your operating system, sometimes there are later versions for one OS, but not for another.

The reason is that between a verion like 3.6 and 3.7, there may be major changes that change how Python works. Generally, there will be backwards compatibility, but some changes may break how some of your packages work. However, when going up a minor version, there won't be any such breaking changes, just fixes and additions that don't get in the way of what was already there. A change from 2.x to 3.x only happens if the language itself goes through a major change, and rarely happens (and perhaps never will again, depending on who you ask).

An exception to the "no minor version change problems" is of course if you run some script that very specifically relies on something that was broken in 3.8.6, but no fixed in 3.8.7+ (as an example). However, that's very bad coding, to rely on what's broken and not fixing it later, so only go along with that if you have no other recourse. Otherwise, just the latest minor version of any version you're after.

Also: make sure you pick the correct architecture. If there's no specific requirement, just pick 64-bit, but if your script needs to interact with other installed software at the binary level, it may require you to install 32-bit Python (and 32-bit packages as well). If you have no such requirement, 64-bit allows more memory access and has some other benefits on modern computers.

2. Why does Python have such heavy dependency on the exact versions of libraries and packages etc?

It's not just Python, this is true for many languages. It's just more visible to the end user for Python, because you run it as an interpreted language. It's only compiled at the very last moment, on the computer it's running on.

This has the advantage that the code can run on a variety of computers and operating systems, but the downside that you need the right environment where you're running it. For people who code in languages like C++, they have to deal with this problem when they're coding, but target a much smaller number of environments (although there's still runtimes to contend with, and DirectX versions, etc.). Other languages just roll everything up into the program that's being distributed, while a Python script by itself can be tiny. It's a design choice.

There are a lot of tools to help you automate the process though and well-written packages will make the process quite painless. If you feel Python is very shakey when it comes to this, that's probable to blame on the packages or scripts you're using, not really the language. The only fault of the language is that it makes it very easy for developers to make such a mess for you and make your life hard with getting specific requirements.

Look for alternatives, but if you can't avoid using a specific script or package, once you figure out how to install or use it, document it or better yet, automate it so you don't have to think about it again.

3. Do virtual environments copy all the files from the main Python installation to create a virtual environment and then install specific packages inside it? Isn't that a lot of wasted resources in duplication because almost all projects require there own virtual environment.

Not all of them, but quite a few of them. However, you still need the original installation to be present on the system. Also, you can't pick up a virtual environment and put it somewhere else, not even on the same PC without some careful changes (often better to just recreate it).

You're right that this is a bit wasteful - but this is a difficult choice.

Either Python would be even more complicated, having to manage many different version of packages in a single environment (Java developers will be able to tell you war stories about this, with their dependency management - or wax lyrically about it, once they get it themselves).

Or you get what we have: a bit wasteful, but in the end diskspace is a lot cheaper than your time. And unlike your time, diskspace is almost infinitely expandable.

You can share virtual environments between very similar projects though, but especially if you get your code from someone else, it's best to not have to worry and just give up a few dozen MB for the project. On the upside: you can just delete a virtual environment directory and that pretty much gets rid of the whole things. Some applications like PyCharm may remember that it was once there, but other than that, that's the virtual environment gone.

Multiple Python versions on the same machine?

I think it is totally independent. Just install them, then you have the commands e.g. /usr/bin/python2.5 and /usr/bin/python2.6. Link /usr/bin/python to the one you want to use as default.

All the libraries are in separate folders (named after the version) anyway.

If you want to compile the versions manually, this is from the readme file of the Python source code:

Installing multiple versions

On Unix and Mac systems if you intend to install multiple versions of Python
using the same installation prefix (--prefix argument to the configure
script) you must take care that your primary python executable is not
overwritten by the installation of a different version. All files and
directories installed using "make altinstall" contain the major and minor
version and can thus live side-by-side. "make install" also creates
${prefix}/bin/python3 which refers to ${prefix}/bin/pythonX.Y. If you intend
to install multiple versions using the same prefix you must decide which
version (if any) is your "primary" version. Install that version using
"make install". Install all other versions using "make altinstall".

For example, if you want to install Python 2.5, 2.6 and 3.0 with 2.6 being
the primary version, you would execute "make install" in your 2.6 build
directory and "make altinstall" in the others.

---

Related Topics