What Is a Virtualenv, and Why Should I Use One

What is a virtualenv, and why should I use one?

Running with the system Python and libraries limits you to one specific Python version, chosen by your OS provider. Trying to run all Python applications on one Python installation makes it likely that version conflicts will occur among the collection of libraries. It's also possible that changes to the system Python will break other OS features that depend on it.

Virtual environments, or "virtualenvs" are lightweight, self-contained Python installations, designed to be set up with a minimum of fuss, and to "just work" without requiring extensive configuration or specialized knowledge.

virtualenv avoids the need to install Python packages globally. When a virtualenv is active, pip will install packages within the environment, which does not affect the base Python installation in any way.

In Python 3.3 or later, you can create a virtualenv as follows:

$ python3 -m venv ENV_DIR

For Windows, you should replace python3 with the full path to python.exe:

>C:\Python34\python.exe -m venv ENV_DIR

(This is a typical Python installation; your system may vary.)

In older versions of Python, including Python 2, one of the following commands should work in most cases:

$ virtualenv ENV_DIR
$ venv ENV_DIR
$ pyvenv ENV_DIR
$ pyvenv3 ENV_DIR

ENV_DIR should be a non-existent directory. The directory can have any name, but to keep these instructions simple, I will assume you have created your virtualenv in a directory called venv (e.g. with python3 -m venv ./venv).

To work in your virtualenv, you activate it:

$ . ./venv/bin/activate
(venv)$ 

Or use this if you have a windows system:

$ venv\Scripts\activate

The (venv) in the shell prompt lets you know which virtualenv you have activated, but you can turn this feature off if you do not like it. You can run all the usual Python commands, and they will be local to your virtualenv:

(venv)$ pip install requests numpy
[...]
(venv)$ python
[...]
>>> import requests
>>> import numpy as np
>>> 

python will run the version of Python that you installed into your virtualenv, so (for example) you don't have to type python3 to get Python 3. The Python that it runs will have access to all the standard library modules and all the packages you installed into the virtualenv, but (by default) none of the packages installed in the system-wide site-packages directory.

This last rule is important: by restricting your virtualenv to only use locally-installed packages, you can ensure that you control exactly which dependencies your project is using, even if some new system-wide package gets installed or updated next week. If you like, you can get a listing of your installed packages:

(venv)$ pip freeze
requests==2.13.0
numpy==1.12.0
(venv)$ 

pip can also parse this format and install from it, and it will install the same versions, even if updates have been released in the meantime:

(venv)$ pip freeze >requirements.txt

(some-other-venv)$ pip install -r requirements.txt
[...]
(some-other-venv)$ python
>>> import requests
>>> import numpy as np
>>> 

You can get out of the virtualenv by deactivating it:

(venv)$ deactivate
$ python
[...]
>>> import requests
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: No module named 'requests'

You can create as many virtualenvs as you like, and they won't interfere with each other, nor with your system packages. A virtualenv is "just" a directory with a bunch of binaries and scripts under it, so you can remove a virtualenv the same way you remove any directory (rm -r venv on Unix). If the virtualenv is activated when you remove it, you may confuse your shell, so it's probably a good idea to deactivate first in that case.

Why is virtualenv necessary?

Virtualenv keeps your Python packages in a virtual environment localized to your project, instead of forcing you to install your packages system-wide.

There are a number of benefits to this,

• the first and principle one is that you can have multiple virtulenvs, so you
  can have multiple sets of packages that for different projects, even
  if those sets of packages would normally conflict with one another.
  For instance, if one project you are working on runs on Django 1.4
  and another runs on Django 1.6, virtualenvs can keep those projects
  fully separate so you can satisfy both requirements at once.
• the second, make it easy for you to release your project with its own dependent
  modules.Thus you can make it easy to create your requirements.txt
  file.
• the third, is that it allows you to switch to another installed python interpreter for that project*. Very useful (Think old 2.x scripts), but sadly not available in the now built-in venv.

Note that virtualenv is about "virtual environments" but is not the same as "virtualization" or "virtual machines" (this is confusing to some). For instance, VMWare is totally different from virtualenv.

Why do we need Virtual Environment in Django?

You can see about python venv in PEP 405: https://www.python.org/dev/peps/pep-0405/

And there is some reasons mentioned in https://realpython.com/python-virtual-environments-a-primer/#why-the-need-for-virtual-environments:

Consider the following scenario where you have two projects: ProjectA and ProjectB, both of which have a dependency on the same library, ProjectC. The problem becomes apparent when we start requiring different versions of ProjectC. Maybe ProjectA needs v1.0.0, while ProjectB requires the newer v2.0.0, for example.
This is a real problem for Python since it can’t differentiate between versions in the site-packages directory. So both v1.0.0 and v2.0.0 would reside in the same directory with the same name:
/System/Library/Frameworks/Python.framework/Versions/3.5/Extras/lib/python/ProjectC
Since projects are stored according to just their name, there is no differentiation between versions. Thus, both projects, ProjectA and ProjectB, would be required to use the same version, which is unacceptable in many cases.

What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, pipenv, etc?

This is my personal recommendation for beginners: start by learning virtualenv and pip, tools which work with both Python 2 and 3 and in a variety of situations, and pick up other tools once you start needing them.

Now on to the answer to the question: what is the difference between these simalarly named things: venv, virtualenv, etc?

PyPI packages not in the standard library:

• virtualenv is a very popular tool that creates isolated Python environments for Python libraries. If you're not familiar with this tool, I highly recommend learning it, as it is a very useful tool.

  It works by installing a bunch of files in a directory (eg: env/), and then modifying the PATH environment variable to prefix it with a custom bin directory (eg: env/bin/). An exact copy of the python or python3 binary is placed in this directory, but Python is programmed to look for libraries relative to its path first, in the environment directory. It's not part of Python's standard library, but is officially blessed by the PyPA (Python Packaging Authority). Once activated, you can install packages in the virtual environment using pip.

• pyenv is used to isolate Python versions. For example, you may want to test your code against Python 2.7, 3.6, 3.7 and 3.8, so you'll need a way to switch between them. Once activated, it prefixes the PATH environment variable with ~/.pyenv/shims, where there are special files matching the Python commands (python, pip). These are not copies of the Python-shipped commands; they are special scripts that decide on the fly which version of Python to run based on the PYENV_VERSION environment variable, or the .python-version file, or the ~/.pyenv/version file. pyenv also makes the process of downloading and installing multiple Python versions easier, using the command pyenv install.

• pyenv-virtualenv is a plugin for pyenv by the same author as pyenv, to allow you to use pyenv and virtualenv at the same time conveniently. However, if you're using Python 3.3 or later, pyenv-virtualenv will try to run python -m venv if it is available, instead of virtualenv. You can use virtualenv and pyenv together without pyenv-virtualenv, if you don't want the convenience features.

• virtualenvwrapper is a set of extensions to virtualenv (see docs). It gives you commands like mkvirtualenv, lssitepackages, and especially workon for switching between different virtualenv directories. This tool is especially useful if you want multiple virtualenv directories.

• pyenv-virtualenvwrapper is a plugin for pyenv by the same author as pyenv, to conveniently integrate virtualenvwrapper into pyenv.

• pipenv aims to combine Pipfile, pip and virtualenv into one command on the command-line. The virtualenv directory typically gets placed in ~/.local/share/virtualenvs/XXX, with XXX being a hash of the path of the project directory. This is different from virtualenv, where the directory is typically in the current working directory. pipenv is meant to be used when developing Python applications (as opposed to libraries). There are alternatives to pipenv, such as poetry, which I won't list here since this question is only about the packages that are similarly named.

Standard library:

• pyvenv (not to be confused with pyenv in the previous section) is a script shipped with Python 3.3 to 3.7. It was removed from Python 3.8 as it had problems (not to mention the confusing name). Running python3 -m venv has exactly the same effect as pyvenv.

• venv is a package shipped with Python 3, which you can run using python3 -m venv (although for some reason some distros separate it out into a separate distro package, such as python3-venv on Ubuntu/Debian). It serves the same purpose as virtualenv, but only has a subset of its features (see a comparison here). virtualenv continues to be more popular than venv, especially since the former supports both Python 2 and 3.

Do I need virtualenv?

Creating a virtualenv sould be quicker and easier than creating a new user. I would not recomend switching for existing projects, but consider it for new projects.

What is the relationship between virtualenv and pyenv?

Pyenv and virtualenv are very different tools that work in different ways to do different things:

• Pyenv is a bash extension - will not work on Windows - that intercepts your calls to python, pip, etc., to direct them to one of several of the system python tool-chains. So you always have all the libraries that you have installed in the selected python version available - as such it is good for users who have to switch between different versions of python.

• VirtualEnv, is pure python so works everywhere, it makes a copy of, optionally a specific version of, python and pip local to the activate environment which may or may not include links to the current system tool-chain, if it does not you can install just a known subset of libraries into that environment. As such it is almost certainly much better for testing and deployment as you know exactly which libraries, at which versions, are used and a global change will not impact your module.

venv python > 3.3

Note that from Python 3.3 onward there is a built in implementation of VirtualEnv called venv (with, on some installations a wrapper called pyvenv - this wrapper is deprecated in Python 3.6), which should probably be used in preference. To avoid possible issues with the wrapper it is often a good idea to use it directly by using /path/to/python3 -m venv desired/env/path or you can use the excellent py python selector on windows with py -3 -m venv desired/env/path. It will create the directory specified with desired/env/path configure and populate it appropriately. In general it is very much like using VirtualEnv.

Additional Tools

There are a number of tools that it is worth mentioning, and considering, as they can help with the use of one or more of the above:

• VirtualEnvWrapper Manage and simplify the use and management of VirtualEnv - Cross Platform.
• pyenv-virtualenv, installed by pyenv-installer, which gives PyEnv tools for managing and interfacing to VirtualEnv - with this you can have a base installation that includes more than one version of python and create isolated environments within each of them - Linux/OS-X. Suggested by Johann Visagie
• PyInstaller can take your python code, possibly developed & tested under VirtualEnv, and bundle it up so that it can run one platforms that do not have your version of python installed - Note that it is not a cross compiler you will need a Windows (virtual-)machine to build Windows installs, etc., but it can be handy even where you can be sure that python will be installed but cannot be sure that the version of python and all the libraries will be compatible with your code.

Does Conda replace the need for virtualenv?

1. Conda replaces virtualenv. In my opinion it is better. It is not limited to Python but can be used for other languages too. In my experience it provides a much smoother experience, especially for scientific packages. The first time I got MayaVi properly installed on Mac was with conda.

2. You can still use pip. In fact, conda installs pip in each new environment. It knows about pip-installed packages.

For example:

conda list

lists all installed packages in your current environment.
Conda-installed packages show up like this:

sphinx_rtd_theme          0.1.7                    py35_0    defaults

and the ones installed via pip have the <pip> marker:

wxpython-common           3.0.0.0                   <pip>

Why do i need to use a virtual envirotment with django?

You don't, but it's nice to have. If you plan on working on other python projects or having other people work on it, it'll probably be a good idea to have it..

then to set up the project in any pc all you need is:

virtualenv django
workon django
pip install -r requirements.txt
# ^ remember to make a req! it's super nice
# init the db
python manage.py runserver

Super easy!

Note: if you use windows you need the virtualenvwrapper package

---

Related Topics