Skip to content

Commit

Permalink
Merge pull request #1416 from Czaki/poerty_problem
Browse files Browse the repository at this point in the history
  • Loading branch information
joerick authored Jul 9, 2023
2 parents 740f081 + f44d278 commit 7f6fe8a
Showing 1 changed file with 34 additions and 20 deletions.
54 changes: 34 additions & 20 deletions docs/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,61 +20,75 @@ Linux wheels are built in [`manylinux`/`musllinux` containers](https://github.co

### Building macOS wheels for Apple Silicon {: #apple-silicon}

`cibuildwheel` supports cross-compiling `universal2` and `arm64` wheels on `x86_64` runners. With the introduction of Apple Silicon, you now have several choices for wheels for Python 3.8+:
`cibuildwheel` supports cross-compiling `universal2` and `arm64` wheels on `x86_64` runners.

#### `x86_64`
#### Overview of Mac architectures

With the introduction of Apple Silicon, you now have several choices for wheels for Python 3.8+:

##### `x86_64`

The traditional wheel for Apple, loads on Intel machines, and on
Apple Silicon when running Python under Rosetta 2 emulation.

Due to a change in naming, Pip 20.3+ (or an installer using packaging 20.5+)
is required to install a binary wheel on macOS Big Sur.

#### `arm64`
##### `arm64`

The native wheel for macOS on Apple Silicon.

Requires Pip 20.3+ (or packaging 20.5+) to install.

#### `universal2`
##### `universal2`

This wheel contains both architectures, causing it to be up to twice the
size (data files do not get doubled, only compiled code). It requires
Pip 20.3 (Packaging 20.6+) to load on Intel, and Pip 21.0.1 (Packaging 20.9+)
to load on Apple Silicon.

!!! note
The dual-architecture `universal2` has a few benefits, but a key benefit
to a universal wheel is that a user can bundle these wheels into an
application and ship a single binary.
The dual-architecture `universal2` has a few benefits, but a key benefit
to a universal wheel is that a user can bundle these wheels into an
application and ship a single binary.

However, if you have a large library, then you might prefer to ship
the two single-arch wheels instead - `x86_64` and `arm64`. In rare cases,
you might want to build all three, but in that case, pip will not download
the universal wheels, because it prefers the most specific wheel
available.
However, if you have a large library, then you might prefer to ship
the two single-arch wheels instead - `x86_64` and `arm64`. In rare cases,
you might want to build all three, but in that case, pip will not download
the universal wheels, because it prefers the most specific wheel
available.

#### What to provide?

Generally speaking, because Pip 20.3 is required for the `universal2` wheel,
most packages should provide both `x86_64` and one of `universal2`/`arm64`
wheels for now. When Pip 20.3+ is common on macOS, then it might be possible
to ship only the `universal2` wheel.
wheels. When Pip 20.3+ is common on macOS, then it might be possible to ship
only the `universal2` wheel.

Opinions vary on which of arch-specific or `universal2` wheels are best - some packagers prefer `universal2` because it's one wheel for all Mac users, so simpler, and easier to build into apps for downstream users. However, because they contain code for both architectures, their file size is larger, meaning they consume more disk space and bandwidth, and are harder to build for some projects.

See [GitHub issue 1333](https://github.com/pypa/cibuildwheel/issues/1333) for more discussion.

**Apple Silicon wheels are not built by default on Intel runners**, but can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `x86_64 arm64`. Cross-compilation is provided by the Xcode toolchain.
#### How to cross-compile

With the exception of Cirrus CI, macOS CI runners are still Intel-based, and Apple Silicon wheels are not built by default on Intel runners. However, cross-compilation can be enabled by adding extra archs to the [`CIBW_ARCHS_MACOS` option](options.md#archs) - e.g. `x86_64 arm64`. Cross-compilation is provided by the Xcode toolchain.

!!! important
When cross-compiling on Intel, it is not possible to test `arm64` and the `arm64` part of a `universal2` wheel.

`cibuildwheel` will raise a warning to notify you of this - these warnings be be silenced by skipping testing on these platforms: `CIBW_TEST_SKIP: *_arm64 *_universal2:arm64`.

Hopefully, cross-compilation is a temporary situation. Once we have widely
available Apple Silicon CI runners, we can build and test `arm64` and
`universal2` wheels natively. That's why `universal2`/`arm64` wheels require opt-in by setting `CIBW_ARCHS_MACOS`.

!!! note
Your runner needs Xcode Command Line Tools 12.2 or later to build `universal2` or `arm64`.

Only CPython 3.8 and newer support `universal2` and `arm64` wheels.

!!! note
If your project uses **Poetry** as a build backend, cross-compiling on macOS [does not currently work](https://github.com/python-poetry/poetry/issues/7107). In some cases arm64 wheels can be built but their tags will be incorrect, with the platform tag showing `x86_64` instead of `arm64`.

As a workaround, the tag can be fixed before running delocate to repair the wheel. The [`wheel tags`](https://wheel.readthedocs.io/en/stable/reference/wheel_tags.html) command is ideal for this. See [this workflow](https://gist.github.com/anderssonjohan/49f07e33fc5cb2420515a8ac76dc0c95#file-build-pendulum-wheels-yml-L39-L53) for an example usage of `wheel tags`.

Hopefully, cross-compilation is a temporary situation. Once we have widely available Apple Silicon CI runners, we can build `arm64` wheels natively. Until then, cross-compiling `arm64` or `universal2` wheels requires opt-in by setting `CIBW_ARCHS_MACOS` on Intel runners.

Here's an example GitHub Actions workflow with a job that builds for Apple Silicon:

> .github/workflows/build_macos.yml
Expand Down

0 comments on commit 7f6fe8a

Please sign in to comment.