Difference Between Pip and Conda

What is the difference between pip and conda?

Quoting from the Conda blog:

Having been involved in the python world for so long, we are all aware of pip, easy_install, and virtualenv, but these tools did not meet all of our specific requirements. The main problem is that they are focused around Python, neglecting non-Python library dependencies, such as HDF5, MKL, LLVM, etc., which do not have a setup.py in their source code and also do not install files into Python’s site-packages directory.

So Conda is a packaging tool and installer that aims to do more than what pip does; handle library dependencies outside of the Python packages as well as the Python packages themselves. Conda also creates a virtual environment, like virtualenv does.

As such, Conda should be compared to Buildout perhaps, another tool that lets you handle both Python and non-Python installation tasks.

Because Conda introduces a new packaging format, you cannot use pip and Conda interchangeably; pip cannot install the Conda package format. You can use the two tools side by side (by installing pip with conda install pip) but they do not interoperate either.

Since writing this answer, Anaconda has published a new page on Understanding Conda and Pip, which echoes this as well:

This highlights a key difference between conda and pip. Pip installs Python packages whereas conda installs packages which may contain software written in any language. For example, before using pip, a Python interpreter must be installed via a system package manager or by downloading and running an installer. Conda on the other hand can install Python packages as well as the Python interpreter directly.

and further on

Occasionally a package is needed which is not available as a conda package but is available on PyPI and can be installed with pip. In these cases, it makes sense to try to use both conda and pip.

Specific reasons to favor pip vs. conda when installing Python packages

I find I use conda first simply because it installs the binary, than try pip if the package isn't there. For instance psycopg2 is far easier to install in conda than pip.

https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

Pip, which stands for Pip Installs Packages, is Python's officially-sanctioned package manager, and is most commonly used to install packages published on the Python Package Index (PyPI). Both pip and PyPI are governed and supported by the Python Packaging Authority (PyPA).

In short, pip is a general-purpose manager for Python packages; conda is a language-agnostic cross-platform environment manager. For the user, the most salient distinction is probably this: pip installs python packages within any environment; conda installs any package within conda environments. If all you are doing is installing Python packages within an isolated environment, conda and pip+virtualenv are mostly interchangeable, modulo some difference in dependency handling and package availability. By isolated environment I mean a conda-env or virtualenv, in which you can install packages without modifying your system Python installation.

If we focus on just installation of Python packages, conda and pip serve different audiences and different purposes. If you want to, say, manage Python packages within an existing system Python installation, conda can't help you: by design, it can only install packages within conda environments. If you want to, say, work with the many Python packages which rely on external dependencies (NumPy, SciPy, and Matplotlib are common examples), while tracking those dependencies in a meaningful way, pip can't help you: by design, it manages Python packages and only Python packages.

Conda and pip are not competitors, but rather tools focused on different groups of users and patterns of use.

Is that a bad idea to use conda and pip install on the same environment?

Don't mix conda install and pip install within conda environment. Probably, decide to use conda or virtualenv+piponce and for all. And here is how you decide which one suits you best:

Conda installs various (not only python) conda-adopted packages within conda environment. It gets your environments right if you are into environments.
Pip installs python packages within Python environment (virtualenv is one of them). It gets your python packages installed right.

Safe way to use conda: don't rush for the latest stuff and stick to the available packages and you'll be fine.

Safe way to use pip+virtualenv: if you see a dependency issue or wish to remove and clean up after package - don't. Just burn the house, abandon your old environment and create a new one. One command line and 2-5 minutes later things gonna be nice and tidy again.

Pip is the best tool for installing Python packages among the two of them. Since pip packages normally come out first and only later are adopted for conda (by conda staff or contributors). Chances are, after updating or installing the latest version of Python some of the packages would only be available through pip. And the latest freshest versions of packages would only be available in pip. And mixing pip and conda packages together can be a nightmare (at least if you want to utilize conda's advantages).

Conda is the best when it comes to managing dependencies and replicating environments. When uninstalling a package conda can properly clean up after itself and has better control over conflicting dependency versions. Also, conda can export environment config and, if the planets are right at the moment and the new machine is not too different, replicate that environment somewhere else. Also, conda can have larger control over the environment and can, for example, have a different version of Python installed inside of it (virtualenv - only the Python available in the system). You can always create a conda package when you have no freedom of choosing what to use.

Some relevant facts:

Conda takes more space and time to setup
Conda might be better if you don't have admin rights on the system
Conda will help when you have no system Python
virtualenv+pip will free you up of knowing lots of details like that

Some outdated notions:

Conda used to be better for novice developers back in the day (2012ish). There is no usability gap anymore
Conda was linked to Continuum Analytics too much. Now Conda itself is open source, the packages - not so much.

I'm confused about the difference between pip and pip3, in anaconda env

a package is installed under pip3, but it cannot be found under Python3. Why?

Because you have many different Pythons. pip doesn't install packages for all Pythons; pip3 doesn't install packages for Python3. They install packages for that particular Pythons they're running under. You cannot expect to install a package with one Python and import it in another eve if they're of the same version.

To see what Python is used with a particular pip see its shebang:

head -1 $(which pip)
head -1 $(which pip3)

If the shebang is #!/usr/bin/env python continue investigating with which python (or which python3).

Finding the Python run python -m site to see where from the packages are imported.

Conda vs. pip under Spyder

Part 1: yes I can use conda to create VE and pip to install packages

conda create PR_venv
conda activate PR_venv
conda install pip
pip install --editable .
conda list

The last line shows which packages are installed by conda and which by pip (shown as pypi)

Part 2: spyder by default cannot see the packages. Need to do two things:

conda install spyder-kernels

Open Spyder and Tools > Preferences > Python Interpreter > Use the following interpreter > [full path to VE python command]

Restart Spyder. Now it can see the packages.

(Edit:) this link is great: https://github.com/spyder-ide/spyder/wiki/Working-with-packages-and-environments-in-Spyder