Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support building x86-64 wheel on arm64 macOS machine #114

Merged
merged 2 commits into from
Feb 15, 2021
Merged

Support building x86-64 wheel on arm64 macOS machine #114

merged 2 commits into from
Feb 15, 2021

Conversation

messense
Copy link
Member

This makes arch -x86_64 python3.9 setup.py build_ext works on Apple Silicon Macs.

cc #108 @ronaldoussoren

Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this. Gotta admit I'm not a macOS user / expert so have been reading up as best I can to give this a review...

Also please add a CHANGELOG entry 😃

Comment on lines 108 to 110
if platform.machine() == "arm64" and platform.processor() == "i386":
# arm64 macOS targeting x86_64
target_triple = "x86_64-apple-darwin"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can allow target to be either way around? e.g. depending on the output of platform.processor() we could allow either x86_64-apple-darwin or aarch64-apple-darwin

... also should it be x86_64 instead of i386 ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps we can allow target to be either way around?

I don't think so. Only arch -x86_64 ... works on arm64 macOS, arch -arm64 ... doesn't work on x86_64 macOS.

also should it be x86_64 instead of i386 ?

I've checked on arm64 macOS with arch -x86_64 python3.9 setup.py build_ext it says i386, but I'm not sure whether i386 is the only possible value or not.

Copy link
Member Author

@messense messense Feb 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've also tried using sysctl.proc_translated, but it doesn't seem to work for us.

x86_64 emulation, ret should be 1 but got 0

❯ arch -x86_64 python -m IPython
Python 3.9.1 (default, Jan 14 2021, 04:46:20)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import platform

In [2]: platform.machine()
Out[2]: 'arm64'

In [3]: platform.processor()
Out[3]: 'i386'

In [4]: import ctypes

In [5]:
   ...: from ctypes import cdll, c_char_p, c_void_p, c_size_t, c_int, POINTER, byref
   ...:
   ...: libc = cdll.LoadLibrary("libc.dylib")
   ...: libc.sysctlbyname.argtypes = [c_char_p, c_void_p, POINTER(c_size_t), c_void_p, c_size_t]
   ...: libc.sysctlbyname.restype = c_int

In [6]: ret = c_size_t(0)

In [7]: size = c_size_t(ctypes.sizeof(ret))

In [8]: libc.sysctlbyname(b"sysctl.proc_translated", byref(ret), byref(size), None, 0)
Out[8]: 0

In [9]: ret
Out[9]: c_ulong(0)

native arm64, ret is 0

❯ python -m IPython
Python 3.9.1 (default, Jan 14 2021, 04:46:20)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.20.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import ctypes

In [2]: import platform

In [3]: platform.machine()
Out[3]: 'arm64'

In [4]: platform.processor()
Out[4]: 'arm'

In [5]:
   ...: from ctypes import cdll, c_char_p, c_void_p, c_size_t, c_int, POINTER, byref
   ...:
   ...: libc = cdll.LoadLibrary("libc.dylib")
   ...: libc.sysctlbyname.argtypes = [c_char_p, c_void_p, POINTER(c_size_t), c_void_p, c_size_t]
   ...: libc.sysctlbyname.restype = c_int

In [6]: ret = c_size_t(0)

In [7]: size = c_size_t(ctypes.sizeof(ret))

In [8]: libc.sysctlbyname(b"sysctl.proc_translated", byref(ret), byref(size), None, 0)
Out[8]: 0

In [9]: ret
Out[9]: c_ulong(0)

Copy link
Member Author

@messense messense Feb 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad, sysctlbyname should works with universal2 Python interpreter, I was testing with a arm64 only Python.

Interesting though, the arm64 only Python 3.9.1 was installed with pyenv, while arch -x86_64 python runs fine and results in the weird platform.machine() == 'arm64' and platform.processor() == 'i386', arch -x86_64 /absolute/path/to/python errored with arch: posix_spawnp: /Users/messense/.pyenv/versions/test/bin/python: Bad CPU type in executable

With a universal2 Python interpreter, I get platform.machine() == 'x86_64' and platform.processor() == 'i386' when running it with arch -x86_64 python.

Copy link
Member

@davidhewitt davidhewitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've fixed CHANGELOG merge conflict and will merge once CI has finished.

@davidhewitt davidhewitt merged commit 303fa40 into PyO3:master Feb 15, 2021
@messense messense deleted the arm64-macos-targeting-x86-64 branch February 16, 2021 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants