How to Force a Python Wheel to Be Platform Specific When Building It

How to force a python wheel to be platform specific when building it?

Here's the code that I usually look at from uwsgi

The basic approach is:

setup.py

# ...

try:
    from wheel.bdist_wheel import bdist_wheel as _bdist_wheel
    class bdist_wheel(_bdist_wheel):
        def finalize_options(self):
            _bdist_wheel.finalize_options(self)
            self.root_is_pure = False
except ImportError:
    bdist_wheel = None

setup(
    # ...
    cmdclass={'bdist_wheel': bdist_wheel},
)

The root_is_pure bit tells the wheel machinery to build a non-purelib (pyX-none-any) wheel. You can also get fancier by saying there are binary platform-specific components but no cpython abi specific components.

Create python wheel based on platform

Python projects distributed as wheel do not contain the setup.py file. Therefore it cannot be run at the time of the installation.

The correct way to specify platform specific dependencies for setuptools is the following:

setuptools.setup(
    # ...
    install_requires=[
        "LinuxOnlyDependency ; platform_system=='Linux'",
        "WindowsOnlyDependency ; platform_system=='Windows'"
    ],
    # ...
)

References:

• Section "Declaring platform specific dependencies" of setuptools documentation
• Section "Environment Markers" of PEP 508

How to add platform-specific package data in setup.py?

This is the solution I am currently using for pypdfium2:

• Create a class of supported platforms whose values correspond to the data directory names:

class PlatformNames:
    darwin_x64  = "darwin_x64"
    linux_x64   = "linux_x64"
    windows_x64 = "windows_x64"
    # ...
    sourcebuild = "sourcebuild"

• Wrap setuptools.setup() with a function that takes the platform name as argument and copies platform-dependent files into the source tree as required:

# A list of non-python file names to consider for inclusion in the installation, e. g.
Libnames = (
    "somelib.so",
    "somelib.dll",
    "somelib.dylib",
)

# _clean() removes possible old binaries/bindings
# _copy_bindings() copies the new stuff into the source tree
# _get_bdist() returns a custom `wheel.bdist_wheel` subclass with the `get_tag()` and `finalize_options()` functions overridden so as to tag the wheels according to their target platform.

def mkwheel(pl_name):
    _clean()
    _copy_bindings(pl_name)
    setuptools.setup(
        package_data = {"": Libnames},
        cmdclass = {"bdist_wheel": _get_bdist(pl_name)},
        # ...
    )
    # not cleaning up afterwards so that editable installs work (`pip3 install -e .`)

• In setup.py, query for a custom environment variable defining the target platform (e. g. $PYP_TARGET_PLATFORM).
  
  • If set to a value that indicates the need for a source distribution (e. g. sdist), run the raw setuptools.setup() function without copying in any build artifacts.
  • If set to a platform name, build for the requested platform. This makes packaging platform-independent and avoids the need for native hosts to craft the wheels.
  • If not set, detect the host platform using sysconfig.get_platform() and call mkwheel() with the corresponding PlatformNames member.
    
    • In case the detected platform is not supported, trigger code that performs a source build, moves the created files into data/sourcebuild/ and runs mkwheel(PlatformNames.sourcebuild).
• Write a script that iterates through the platform names, sets your environment variable and runs python3 -m build --no-isolation --skip-dependency-check --wheel for each. Also invoke build once with --sdist instead of --wheel and the environment variable set to the value for source distribution.

→ If all goes well, the platform-specific wheels and a source distribution will be written into dist/.

Perhaps this is a lot easier to understand just by looking at pypdfium2's code (especially setup.py, setup_base.py and craft_packages.py).

Disclaimer: I am not experienced with the setup infrastructure of Python and merely wrote this code out of personal need. I acknowledge that the approach is a bit "hacky". If there is a possibility to achieve the same goal while using the setuptools API in a more official sort of way, I'd be interested to hear about it.

Update 1: A negative implication of this concept is that the content wrongly ends up in a purelib folder, although it should be platlib as per PEP 427. I'm not sure how to instruct wheel/setuptools differently. Luckily, this is rather just a cosmetic problem.

Update 2: Found a fix to the purelib problem:

class BinaryDistribution (setuptools.Distribution):
    def has_ext_modules(self):
        return True

setuptools.setup(
    # ...
    distclass = BinaryDistribution,
)

What happens when you build a Python wheel?

Let me explain in simple terms:

If your code is just Python, building a wheel doesn't actually do much work other than creating an archive as you pointed out.

A wheel is more useful when you have some of the functionality in your package implemented in C (which Python allows as a first-class feature). In such cases, anyone installing your package also needs to compile your C code. And if there is lots of such C code, it can take up a lot of time to install your package. This is where a wheel helps - it provides a way of distributing pre-compiled versions of your package so that others can install and use your package relatively quickly.

How to build a universal wheel with pyproject.toml

Add this section into the pyproject.toml:

[tool.distutils.bdist_wheel]
universal = true

Pip install and platform specific wheels

PIP follows the PEP 425 Use recommendations; this stipulates how a binary distribution wheel is selected.

Specifically, pip install will only consider compatible wheels. A wheel compatible with a different platform is not going to be downloaded.

If there are no compatible wheels, but there is a source distribution, then that source distribution is downloaded and compiled locally. If there are no compatible wheels, and no source distribution, installation fails.

Wheels can also be built for pure python projects, at which point they are no longer platform specific; these are called universal wheels. If a project uses optional binary components they can choose to produce both platform-specific wheels (including the compiled binary components specific to a Python ABI version and platform), and a universal wheel with the optional compiled components excluded. An installer can then select the universal version if no compatible binary version is available for the current platform. This is not all that common however, as a universal wheel would be preferred over a source distribution!

How to specify CPU Architecture of a Python package in setup.py?

Coudn't make it "Win 10 Only", but for "Windows Only", it can be done by using --plat-name argument to setup.py

For 32 bit Windows :-

python setup.py bdist_wheel --plat-name win32

For 64 bit Windows :-

python setup.py bdist_wheel --plat-name win_amd64

Also See : How to force a python wheel to be platform specific when building it?

---

Related Topics