Difference Between Venv, Pyvenv, Pyenv, Virtualenv, Virtualenvwrapper, Pipenv, etc

What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?

This is my personal recommendation for beginners: start by learning virtualenv and pip, tools which work with both Python 2 and 3 and in a variety of situations, and pick up other tools once you start needing them.

Now on to the answer to the question: what is the difference between these simalarly named things: venv, virtualenv, etc?

PyPI packages not in the standard library:

  • virtualenv is a very popular tool that creates isolated Python environments for Python libraries. If you're not familiar with this tool, I highly recommend learning it, as it is a very useful tool.

    It works by installing a bunch of files in a directory (eg: env/), and then modifying the PATH environment variable to prefix it with a custom bin directory (eg: env/bin/). An exact copy of the python or python3 binary is placed in this directory, but Python is programmed to look for libraries relative to its path first, in the environment directory. It's not part of Python's standard library, but is officially blessed by the PyPA (Python Packaging Authority). Once activated, you can install packages in the virtual environment using pip.

  • pyenv is used to isolate Python versions. For example, you may want to test your code against Python 2.7, 3.6, 3.7 and 3.8, so you'll need a way to switch between them. Once activated, it prefixes the PATH environment variable with ~/.pyenv/shims, where there are special files matching the Python commands (python, pip). These are not copies of the Python-shipped commands; they are special scripts that decide on the fly which version of Python to run based on the PYENV_VERSION environment variable, or the .python-version file, or the ~/.pyenv/version file. pyenv also makes the process of downloading and installing multiple Python versions easier, using the command pyenv install.

  • pyenv-virtualenv is a plugin for pyenv by the same author as pyenv, to allow you to use pyenv and virtualenv at the same time conveniently. However, if you're using Python 3.3 or later, pyenv-virtualenv will try to run python -m venv if it is available, instead of virtualenv. You can use virtualenv and pyenv together without pyenv-virtualenv, if you don't want the convenience features.

  • virtualenvwrapper is a set of extensions to virtualenv (see docs). It gives you commands like mkvirtualenv, lssitepackages, and especially workon for switching between different virtualenv directories. This tool is especially useful if you want multiple virtualenv directories.

  • pyenv-virtualenvwrapper is a plugin for pyenv by the same author as pyenv, to conveniently integrate virtualenvwrapper into pyenv.

  • pipenv aims to combine Pipfile, pip and virtualenv into one command on the command-line. The virtualenv directory typically gets placed in ~/.local/share/virtualenvs/XXX, with XXX being a hash of the path of the project directory. This is different from virtualenv, where the directory is typically in the current working directory. pipenv is meant to be used when developing Python applications (as opposed to libraries). There are alternatives to pipenv, such as poetry, which I won't list here since this question is only about the packages that are similarly named.

Standard library:

  • pyvenv (not to be confused with pyenv in the previous section) is a script shipped with Python 3.3 to 3.7. It was removed from Python 3.8 as it had problems (not to mention the confusing name). Running python3 -m venv has exactly the same effect as pyvenv.

  • venv is a package shipped with Python 3, which you can run using python3 -m venv (although for some reason some distros separate it out into a separate distro package, such as python3-venv on Ubuntu/Debian). It serves the same purpose as virtualenv, but only has a subset of its features (see a comparison here). virtualenv continues to be more popular than venv, especially since the former supports both Python 2 and 3.

What's the difference between virtualenv and -m venv in creating Virtual environments(Python)

venv is a package shipped directly with python 3. So you don't need to pip install anything.

virtualenv instead is an independent library available at https://virtualenv.pypa.io/en/stable/ and can be install with pip.

They solve the same problem and work in a very similar manner.

If you use python3 I suggest to avoid an "extra" dependency and just stick with venv.

Your error is probably because you use Python2/pip2.

Conda env vs venv / pyenv / virtualenv / etc

Update 2021-0602: After researching, experiencing, and googling more I found this article. It is detailed, opinionated in what I found a helpful way, and provided everything I was looking for and more. Highly recommend. Conda is quite different from venv.

Original Answer
After researching and playing around, here's what I've found, particularly focused on the difference between conda environments and venv:

  • High level, there's not that much of a difference between conda environments and venv. There are not large performance differences, time in setup differences, replication differences, etc.
  • The decision to use one or the other should primarily by driven by personal preference, and the convention at work (e.g. if your work uses venv for everything, it probably makes sense to use venv and not conda environments.)

There are some differences worth calling out:

  • Conda environments can set up environments for python and also R, so if you switch between the two conda is probably preferable so you only need to learn one set of tools/conventions.
  • Conda environments all get stored in a single folder. This has pros and cons:
  • Pro: you can easily look up all environments you've created.
  • Pro: you can re-use one environment for multiple projects (e.g. I have a "finance" environment which works well for all my finance-related projects.)
  • Con: you have to name all your environments differently, and remember the names (or look them up).
  • Con: it is more of a pain to store that environment in the project folder you've created. This means you need to remember which environment goes with which project, and you can't simply cd into the project folder and then activate the generically named 'env' that is stored in that folder.

For the type of programming I'm doing, I find conda environments helpful. I could easily see use cases where venv is the better choice.

Lastly, Conda is both an environments manager as well as a package manager like PIP. Useful comparison table here.

In short, if you don't have a strong preference already, conda is more robust than venv or pip, can be combined with pip, and is probably the better default option. That said, if you already have a strong preference it means you likely already know how to do what you want, so it's unlikely to be worth it to change.

What is the relationship between virtualenv and pyenv?

Pyenv and virtualenv are very different tools that work in different ways to do different things:

  • Pyenv is a bash extension - will not work on Windows - that intercepts your calls to python, pip, etc., to direct them to one of several of the system python tool-chains. So you always have all the libraries that you have installed in the selected python version available - as such it is good for users who have to switch between different versions of python.

  • VirtualEnv, is pure python so works everywhere, it makes a copy of, optionally a specific version of, python and pip local to the activate environment which may or may not include links to the current system tool-chain, if it does not you can install just a known subset of libraries into that environment. As such it is almost certainly much better for testing and deployment as you know exactly which libraries, at which versions, are used and a global change will not impact your module.

venv python > 3.3

Note that from Python 3.3 onward there is a built in implementation of VirtualEnv called venv (with, on some installations a wrapper called pyvenv - this wrapper is deprecated in Python 3.6), which should probably be used in preference. To avoid possible issues with the wrapper it is often a good idea to use it directly by using /path/to/python3 -m venv desired/env/path or you can use the excellent py python selector on windows with py -3 -m venv desired/env/path. It will create the directory specified with desired/env/path configure and populate it appropriately. In general it is very much like using VirtualEnv.

Additional Tools

There are a number of tools that it is worth mentioning, and considering, as they can help with the use of one or more of the above:

  • VirtualEnvWrapper Manage and simplify the use and management of VirtualEnv - Cross Platform.
  • pyenv-virtualenv, installed by pyenv-installer, which gives PyEnv tools for managing and interfacing to VirtualEnv - with this you can have a base installation that includes more than one version of python and create isolated environments within each of them - Linux/OS-X. Suggested by Johann Visagie
  • PyInstaller can take your python code, possibly developed & tested under VirtualEnv, and bundle it up so that it can run one platforms that do not have your version of python installed - Note that it is not a cross compiler you will need a Windows (virtual-)machine to build Windows installs, etc., but it can be handy even where you can be sure that python will be installed but cannot be sure that the version of python and all the libraries will be compatible with your code.

What is the difference between pyenv, virtualenv, anaconda?

Edit: It's worth mentioning pip here as well, as conda and pip have similarities and differences that are relevant to this topic.

pip: the Python Package Manager.

  • You might think of pip as the python equivalent of the ruby gem command
  • pip is not included with python by default.
  • You may install Python using homebrew, which will install pip automatically: brew install python
  • The final version of OSX did not include pip by default. To add pip to your mac system's version of python, you can sudo easy_install pip
  • You can find and publish python packages using PyPI: The Python Package Index
  • The requirements.txt file is comparable to the ruby gemfile
  • To create a requirements text file, pip freeze > requirements.txt
  • Note, at this point, we have python installed on our system, and we have created a requirements.txt file that outlines all of the python packages that have been installed on your system.

pyenv: Python Version Manager

  • From the docs: pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well. This project was forked from rbenv and ruby-build, and modified for Python.
  • Many folks hesitate to use python3.
  • If you need to use different versions of python, pyenv lets you manage this easily.

virtualenv: Python Environment Manager.

  • From the docs: The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.
  • To create a virtualenv, simply invoke virtualenv ENV, where ENV is is a directory to place the new virtual environment.
  • To initialize the virtualenv, you need to source ENV/bin/activate. To stop using, simply call deactivate.
  • Once you activate the virtualenv, you might install all of a workspace's package requirements by running pip install -r against the project's requirements.txt file.

Anaconda: Package Manager + Environment Manager + Additional Scientific Libraries.

  • From the docs: Anaconda 4.2.0 includes an easy installation of Python (2.7.12, 3.4.5, and/or 3.5.2) and updates of over 100 pre-built and tested scientific and analytic Python packages that include NumPy, Pandas, SciPy, Matplotlib, and IPython, with over 620 more packages available via a simple conda install <packagename>
  • As a web developer, I haven't used Anaconda. It's ~3GB including all the packages.
  • There is a slimmed down miniconda version, which seems like it could be a more simple option than using pip + virtualenv, although I don't have experience using it personally.
  • While conda allows you to install packages, these packages are separate than PyPI packages, so you may still need to use pip additionally depending on the types of packages you need to install.

See also:

  • conda vs pip vs virtualenv (section in documentation from anaconda)
  • the difference between pip and conda (stackoverflow)
  • the relationship between virtualenv and pyenv (stackoverflow)

What is difference between using and not using Python to run the venv command?

According to python docs both are equivalent.
Here is the pvenv script from python 3.4 source code:

#!/usr/bin/env python3
if __name__ == '__main__':
import sys
rc = 1
try:
import venv
venv.main()
rc = 0
except Exception as e:
print('Error: %s' % e, file=sys.stderr)
sys.exit(rc)

Note:

The pyvenv script shipped with Python 3 but has been deprecated in
Python 3.6+ in favour of python3 -m venv. This prevents confusion as
to what Python interpreter pyvenv is connected to and thus what Python
interpreter will be used by the virtual environment.



Related Topics



Leave a reply



Submit