What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?
This is my personal recommendation for beginners: start by learning virtualenv
and pip
, tools which work with both Python 2 and 3 and in a variety of situations, and pick up other tools once you start needing them.
Now on to the answer to the question: what is the difference between these simalarly named things: venv, virtualenv, etc?
PyPI packages not in the standard library:
virtualenv
is a very popular tool that creates isolated Python environments for Python libraries. If you're not familiar with this tool, I highly recommend learning it, as it is a very useful tool.It works by installing a bunch of files in a directory (eg:
env/
), and then modifying thePATH
environment variable to prefix it with a custombin
directory (eg:env/bin/
). An exact copy of thepython
orpython3
binary is placed in this directory, but Python is programmed to look for libraries relative to its path first, in the environment directory. It's not part of Python's standard library, but is officially blessed by the PyPA (Python Packaging Authority). Once activated, you can install packages in the virtual environment usingpip
.pyenv
is used to isolate Python versions. For example, you may want to test your code against Python 2.7, 3.6, 3.7 and 3.8, so you'll need a way to switch between them. Once activated, it prefixes thePATH
environment variable with~/.pyenv/shims
, where there are special files matching the Python commands (python
,pip
). These are not copies of the Python-shipped commands; they are special scripts that decide on the fly which version of Python to run based on thePYENV_VERSION
environment variable, or the.python-version
file, or the~/.pyenv/version
file.pyenv
also makes the process of downloading and installing multiple Python versions easier, using the commandpyenv install
.pyenv-virtualenv
is a plugin forpyenv
by the same author aspyenv
, to allow you to usepyenv
andvirtualenv
at the same time conveniently. However, if you're using Python 3.3 or later,pyenv-virtualenv
will try to runpython -m venv
if it is available, instead ofvirtualenv
. You can usevirtualenv
andpyenv
together withoutpyenv-virtualenv
, if you don't want the convenience features.virtualenvwrapper
is a set of extensions tovirtualenv
(see docs). It gives you commands likemkvirtualenv
,lssitepackages
, and especiallyworkon
for switching between differentvirtualenv
directories. This tool is especially useful if you want multiplevirtualenv
directories.pyenv-virtualenvwrapper
is a plugin forpyenv
by the same author aspyenv
, to conveniently integratevirtualenvwrapper
intopyenv
.pipenv
aims to combinePipfile
,pip
andvirtualenv
into one command on the command-line. Thevirtualenv
directory typically gets placed in~/.local/share/virtualenvs/XXX
, withXXX
being a hash of the path of the project directory. This is different fromvirtualenv
, where the directory is typically in the current working directory.pipenv
is meant to be used when developing Python applications (as opposed to libraries). There are alternatives topipenv
, such aspoetry
, which I won't list here since this question is only about the packages that are similarly named.
Standard library:
pyvenv
(not to be confused withpyenv
in the previous section) is a script shipped with Python 3.3 to 3.7. It was removed from Python 3.8 as it had problems (not to mention the confusing name). Runningpython3 -m venv
has exactly the same effect aspyvenv
.venv
is a package shipped with Python 3, which you can run usingpython3 -m venv
(although for some reason some distros separate it out into a separate distro package, such aspython3-venv
on Ubuntu/Debian). It serves the same purpose asvirtualenv
, but only has a subset of its features (see a comparison here).virtualenv
continues to be more popular thanvenv
, especially since the former supports both Python 2 and 3.
What's the difference between virtualenv and -m venv in creating Virtual environments(Python)
venv is a package shipped directly with python 3. So you don't need to pip install
anything.
virtualenv instead is an independent library available at https://virtualenv.pypa.io/en/stable/ and can be install with pip
.
They solve the same problem and work in a very similar manner.
If you use python3 I suggest to avoid an "extra" dependency and just stick with venv
.
Your error is probably because you use Python2/pip2.
Conda env vs venv / pyenv / virtualenv / etc
Update 2021-0602: After researching, experiencing, and googling more I found this article. It is detailed, opinionated in what I found a helpful way, and provided everything I was looking for and more. Highly recommend. Conda is quite different from venv.
Original Answer
After researching and playing around, here's what I've found, particularly focused on the difference between conda environments and venv:
- High level, there's not that much of a difference between conda environments and venv. There are not large performance differences, time in setup differences, replication differences, etc.
- The decision to use one or the other should primarily by driven by personal preference, and the convention at work (e.g. if your work uses venv for everything, it probably makes sense to use venv and not conda environments.)
There are some differences worth calling out:
- Conda environments can set up environments for python and also R, so if you switch between the two conda is probably preferable so you only need to learn one set of tools/conventions.
- Conda environments all get stored in a single folder. This has pros and cons:
- Pro: you can easily look up all environments you've created.
- Pro: you can re-use one environment for multiple projects (e.g. I have a "finance" environment which works well for all my finance-related projects.)
- Con: you have to name all your environments differently, and remember the names (or look them up).
- Con: it is more of a pain to store that environment in the project folder you've created. This means you need to remember which environment goes with which project, and you can't simply cd into the project folder and then activate the generically named 'env' that is stored in that folder.
For the type of programming I'm doing, I find conda environments helpful. I could easily see use cases where venv is the better choice.
Lastly, Conda is both an environments manager as well as a package manager like PIP. Useful comparison table here.
In short, if you don't have a strong preference already, conda is more robust than venv or pip, can be combined with pip, and is probably the better default option. That said, if you already have a strong preference it means you likely already know how to do what you want, so it's unlikely to be worth it to change.
What is the relationship between virtualenv and pyenv?
Pyenv and virtualenv are very different tools that work in different ways to do different things:
Pyenv is a bash extension - will not work on Windows - that intercepts your calls to python, pip, etc., to direct them to one of several of the system python tool-chains. So you always have all the libraries that you have installed in the selected python version available - as such it is good for users who have to switch between different versions of python.
VirtualEnv, is pure python so works everywhere, it makes a copy of, optionally a specific version of, python and pip local to the activate environment which may or may not include links to the current system tool-chain, if it does not you can install just a known subset of libraries into that environment. As such it is almost certainly much better for testing and deployment as you know exactly which libraries, at which versions, are used and a global change will not impact your module.
venv python > 3.3
Note that from Python 3.3 onward there is a built in implementation of VirtualEnv called venv (with, on some installations a wrapper called pyvenv - this wrapper is deprecated in Python 3.6), which should probably be used in preference. To avoid possible issues with the wrapper it is often a good idea to use it directly by using /path/to/python3 -m venv desired/env/path
or you can use the excellent py
python selector on windows with py -3 -m venv desired/env/path
. It will create the directory specified with desired/env/path
configure and populate it appropriately. In general it is very much like using VirtualEnv.
Additional Tools
There are a number of tools that it is worth mentioning, and considering, as they can help with the use of one or more of the above:
- VirtualEnvWrapper Manage and simplify the use and management of VirtualEnv - Cross Platform.
- pyenv-virtualenv, installed by pyenv-installer, which gives PyEnv tools for managing and interfacing to VirtualEnv - with this you can have a base installation that includes more than one version of python and create isolated environments within each of them - Linux/OS-X. Suggested by Johann Visagie
- PyInstaller can take your python code, possibly developed & tested under VirtualEnv, and bundle it up so that it can run one platforms that do not have your version of python installed - Note that it is not a cross compiler you will need a Windows (virtual-)machine to build Windows installs, etc., but it can be handy even where you can be sure that python will be installed but cannot be sure that the version of python and all the libraries will be compatible with your code.
What is the difference between pyenv, virtualenv, anaconda?
Edit: It's worth mentioning pip
here as well, as conda
and pip
have similarities and differences that are relevant to this topic.
pip: the Python Package Manager.
- You might think of
pip
as the python equivalent of the rubygem
command pip
is not included with python by default.- You may install Python using homebrew, which will install pip automatically:
brew install python
- The final version of OSX did not include pip by default. To add pip to your mac system's version of python, you can
sudo easy_install pip
- You can find and publish python packages using PyPI: The Python Package Index
- The requirements.txt file is comparable to the ruby
gemfile
- To create a requirements text file,
pip freeze > requirements.txt
- Note, at this point, we have python installed on our system, and we have created a requirements.txt file that outlines all of the python packages that have been installed on your system.
pyenv: Python Version Manager
- From the docs: pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well. This project was forked from rbenv and ruby-build, and modified for Python.
- Many folks hesitate to use python3.
- If you need to use different versions of python,
pyenv
lets you manage this easily.
virtualenv: Python Environment Manager.
- From the docs: The basic problem being addressed is one of dependencies and versions, and indirectly permissions. Imagine you have an application that needs version 1 of LibFoo, but another application requires version 2. How can you use both these applications? If you install everything into /usr/lib/python2.7/site-packages (or whatever your platform’s standard location is), it’s easy to end up in a situation where you unintentionally upgrade an application that shouldn’t be upgraded.
- To create a
virtualenv
, simply invokevirtualenv ENV
, whereENV
is is a directory to place the new virtual environment. - To initialize the
virtualenv
, you need tosource ENV/bin/activate
. To stop using, simply calldeactivate
. - Once you activate the
virtualenv
, you might install all of a workspace's package requirements by runningpip install -r
against the project'srequirements.txt
file.
Anaconda: Package Manager + Environment Manager + Additional Scientific Libraries.
- From the docs: Anaconda 4.2.0 includes an easy installation of Python (2.7.12, 3.4.5, and/or 3.5.2) and updates of over 100 pre-built and tested scientific and analytic Python packages that include NumPy, Pandas, SciPy, Matplotlib, and IPython, with over 620 more packages available via a simple
conda install <packagename>
- As a web developer, I haven't used Anaconda. It's ~3GB including all the packages.
- There is a slimmed down
miniconda
version, which seems like it could be a more simple option than usingpip
+virtualenv
, although I don't have experience using it personally. - While
conda
allows you to install packages, these packages are separate than PyPI packages, so you may still need to use pip additionally depending on the types of packages you need to install.
See also:
- conda vs pip vs virtualenv (section in documentation from anaconda)
- the difference between pip and conda (stackoverflow)
- the relationship between virtualenv and pyenv (stackoverflow)
What is difference between using and not using Python to run the venv command?
According to python docs both are equivalent.
Here is the pvenv script from python 3.4 source code:
#!/usr/bin/env python3
if __name__ == '__main__':
import sys
rc = 1
try:
import venv
venv.main()
rc = 0
except Exception as e:
print('Error: %s' % e, file=sys.stderr)
sys.exit(rc)
Note:
The
pyvenv
script shipped with Python 3 but has been deprecated in
Python 3.6+ in favour ofpython3 -m venv
. This prevents confusion as
to what Python interpreter pyvenv is connected to and thus what Python
interpreter will be used by the virtual environment.
Related Topics
Groupby Pandas Dataframe and Select Most Common Value
Multiple Instances of Python Running Simultaneously Limited to 35
"Valueerror: _Type_ 'V' Not Supported" Error After Installing Pyreadline
Differencebetween C.Utf-8 and En_Us.Utf-8 Locales
How to Use Os.Umask() in Python
Install Tkinter on Amazon Linux
Why Is My Python App Stalled with 'System'/Kernel CPU Time
Python - Can't Open Lib 'Libtdsodbc.So':File Not Found
Python List by Value Not by Reference
How to "Watch" a File for Modification/Change
Default Buffer Size for a File on Linux
What Conditions Result in an Opened, Nonblocking Named Pipe (Fifo) Being "Unavailable" for Reads
Finding the Command for a Specific Pid in Linux from Python