Differences Between Distribute, Distutils, Setuptools and Distutils2

Differences between distribute, distutils, setuptools and distutils2?

As of May 2022, most of the other answers to this question are several years out-of-date. When you come across advice on Python packaging issues, remember to look at the date of publication, and don't trust out-of-date information.

The Python Packaging User Guide is worth a read. Every page has a "last updated" date displayed, so you can check the recency of the manual, and it's quite comprehensive. The fact that it's hosted on a subdomain of python.org of the Python Software Foundation just adds credence to it. The Project Summaries page is especially relevant here.

Summary of tools:

Here's a summary of the Python packaging landscape:

Supported tools:

  • setuptools was developed to overcome Distutils' limitations, and is not included in the standard library. It introduced a command-line utility called easy_install. It also introduced the setuptools Python package that can be imported in your setup.py script, and the pkg_resources Python package that can be imported in your code to locate data files installed with a distribution. One of its gotchas is that it monkey-patches the distutils Python package. It should work well with pip. It sees regular releases.

    • Official docs | Pypi page | GitHub repo | setuptools section of Python Package User Guide
  • scikit-build is an improved build system generator that internally uses CMake to build compiled Python extensions. Because scikit-build isn't based on distutils, it doesn't really have any of its limitations. When ninja-build is present, scikit-build can compile large projects over three times faster than the alternatives. It should work well with pip.

    • Official docs | Pypi page | GitHub repo | scikit-build section of Python Package User Guide
  • distlib is a library that provides functionality that is used by higher level tools like pip.

    • Official Docs | Pypi page | Bitbucket repo | distlib section of Python Package User Guide
  • packaging is also a library that provides functionality used by higher level tools like pip and setuptools

    • Official Docs | Pypi page | GitHub repo | packaging section of Python Package User Guide

Deprecated/abandoned tools:

  • distutils is still included in the standard library of Python, but is considered deprecated as of Python 3.10. It is useful for simple Python distributions, but lacks features. It introduces the distutils Python package that can be imported in your setup.py script.

    • Official docs | distutils section of Python Package User Guide
  • distribute was a fork of setuptools. It shared the same namespace, so if you had Distribute installed, import setuptools would actually import the package distributed with Distribute. Distribute was merged back into Setuptools 0.7, so you don't need to use Distribute any more. In fact, the version on Pypi is just a compatibility layer that installs Setuptools.

  • distutils2 was an attempt to take the best of distutils, setuptools and distribute and become the standard tool included in Python's standard library. The idea was that distutils2 would be distributed for old Python versions, and that distutils2 would be renamed to packaging for Python 3.3, which would include it in its standard library. These plans did not go as intended, however, and currently, distutils2 is an abandoned project. The latest release was in March 2012, and its Pypi home page has finally been updated to reflect its death.

Others:

There are other tools, if you are interested, read Project Summaries in the Python Packaging User Guide. I won't list them all, to not repeat that page, and to keep the answer matching the question, which was only about distribute, distutils, setuptools and distutils2.

Recommendation:

If all of this is new to you, and you don't know where to start, I would recommend learning setuptools, along with pip and virtualenv, which all work very well together.

If you're looking into virtualenv, you might be interested in this question: What is the difference between venv, pyvenv, pyenv, virtualenv, virtualenvwrapper, etc?. (Yes, I know, I groan with you.)

How do I use setuptools or distutils to distribute a script as opposed to a Python package?

You should use setuptools entry points and pip install pkg will create a bin/ script for you. When you do a system-wide package installation the script will go to /usr/bin or /usr/local/bin.

About entry points

How do setuptools, distribute, and pip relate to one another?

[2014-10 TL;DR:
distribute is dead, use pip, the new setuptools, and, for binary distributions, wheels. More below.]


[Original answer]

Distribute is was a fork of the older setuptools so nearly all comments that follow apply equally to Distribute and setuptools. Setuptools was an attempt to fill in a number of holes in the even older Python standard library package, Distutils. Setuptools added functions like automatic downloads of packages via a command-line interface, easy_install, and some level of dependency management. However, a segment of the Python community is of the opinion that setuptools is too intrusive and has too much behind-the-scenes magic for some of its features.

pip is a higher-level interface on top of setuptools or Distribute. It uses them to perform many of its functions but avoids some of their more controversial features, like zipped eggs. pip also provides features not available in setuptools, like an uninstall command and the ability to define fixed sets of requirements and reliably reproduce a set of packages. There is a more complete feature comparison here.

Why are there so many components (and there are more, like buildout)? Lots of reasons: solutions must work across all of the major platforms on which Python is supported (i.e. Unix-y, Windows, Mac OS X), so building and installation present a complex set of problems; like many open-source projects, Python is essentially all-volunteer and many developers just aren't all that interested in packaging and installation issues; there is a natural conservatism about adding major new unproven features to the standard library; differences in opinions, etc etc.

At the moment, there is a project underway to provide a replacement for Distutils and possibly for some of the higher-level add-ons. It is planned to be released in the Python 3.3 standard library as the packaging package and as an add-on for older versions of Python as Distutils2.

To summarize, the current relationship is:

pip -> [ setuptools | Distribute ] -> Distutils -> Python core
|
3rd party packages | included in Python
|

UPDATE (2012-07): Prior to feature code cutoff for Python 3.3, it was decided that packaging was not quite ready yet for release in the standard library so it has been removed from the 3.3 release. Work will continue on Distutils2 which is available via PyPI and on what will be included in the standard library for Python 3.4.


UPDATE (2014-10): There have been further changes in the world of Python packaging since this answer was last updated.

  • Most importantly, since mid-2013, the rift between setuptools and
    distribute has been healed and development activity has been merged
    into a new setuptools project. distribute is now
    deprecated and no longer maintained; use the new setuptools instead
    but don't use its easy_install as an installer.

  • pip has become the de-facto and blessed installer tool (for Python
    packages not otherwise provided by your platform's package manager)
    either in- or outside of virtual environments (virtualenv or pyvenv).

  • Instead of the old setuptools bdist eggs, wheels have
    become the blessed binary distribution format for Python packages.

  • As of Python 3.4, a version of pip with wheel support
    is now shipped with the official python.org binary installers and
    source packages and it is anticipated that pip will also be
    included in the next maintenance release of Python 2.7 (2.7.9).

  • Distutils2 and packaging are now dormant.

More details in the new Distributing Python Modules section of the Python 3 docs and the new Python Packaging User Guide.

What should I install Distribute or Setuptools

The situation is legitimately confusing as there are too many installers available for Python and the landscape has changed recently.

Distribute was a fork of setuptools which itself is an extension to distutils. They merged back with setuptools in 2013. Your book is most likely out of date. The documentation of setuptools and distribute has been a confusing mess since it assumes you already have intimate knowledge of distutils. Distutils2 was an abandoned effort to get a more capable distutils into the Py3.3 standard lib.

Since distutils still lacks key features like generating executable wrapper scripts you would be best off working with a recent version of setuptools. Read through the distutils documentation first as setuptools is a superset of its functionality.

You can't depend on your users having setuptools installed so it is helpful to include the ez_setup.py bootstrapping script with your code. This will let your setup.py install setuptools if needed.

Distribute/distutils specify Python version

Not sure if there's some special setting, but this in the beginning of setup.py might help:

import sys
if sys.version_info.major < 3:
print("I'm only for 3, please upgrade")
sys.exit(1)


Related Topics



Leave a reply



Submit