Implement Basic Cross Compile Support

[mtp401]

This PR implements basic cross compile support. When cross compiling, the target python interpreter cannot be invoked since it may be built for an incompatible architecture (e.g. an armv7 target on an x86_64 host). Instead, attempt to parse the target interpreters compilation options out of it's headers.

[kngwyu]

Thanks!
But I'm sorry I don't know what the patchlevel.h is and what parse_header_defines does.
Could you please explain them more?

[mtp401]

I think the comment for parse_header_defines sums up the behavior of that function. It takes a path to a C header and parses every single line #define X Y statement into a HashMap entry that maps X to Y. patchlevel.h is where cpython keeps all of the versioning information associated with the python interpreter. You can take a look at patchlevel.h for Python 3.8.0 as an example.

[kngwyu]

Thanks!
I still don't like the situation we need the header file only to know the python configs, but other than that, I'm 👍 for this PR.

[konstin]

Thank you for your pull request!

Please add some documentation about how to use cross compiling with pyo3, ideally with instructions on installing the required software. Please move the documentation to a chapter in the book. Some instructions about required software and cargo invocations would also be helpful.

---

As far as I can see, the only mandatory information from the header files is the major and the minor version. Would it be possible to pass the major and minor version through environment variables or features, so you wouldn't even need to have the python version installed to build a native module? This is nothing I expect you to implement, but something that could be very useful even for same-platform compilation.

[mtp401]

As for:

As far as I can see, the only mandatory information from the header files is the major and the minor version. Would it be possible to pass the major and minor version through environment variables or features, so you wouldn't even need to have the python version installed to build a native module? This is nothing I expect you to implement, but something that could be very useful even for same-platform compilation.

There are other important pieces of information that are parsed out of pyconfig.h that are dependent not only on the version of Python itself, but also how the interpreter was compiled (Py_UNICODE_SIZE is a good example of this). Additionally, wouldn't you still needed Python installed as you need to link against the interpreter eventually anyway?

[konstin]

Python 2 there are actually two versions Py_UNICODE_SIZE used and many packages are therefore build for 27m and 27mu. For python 3 however there seems to be one default abi, with python 3.3 even removing the ufor Py_UNICODE_SIZE from the abi. This means that you need those constants for specific python builds (e.g. when you're compiling against a debug build of python), but there's a proper set of defaults we could use otherwise.

Additionally, wouldn't you still needed Python installed as you need to link against the interpreter eventually anyway?

For bin crates, yes, but afaik not for cdylibs. My thought was mostly that it would be super useful if you could build wheels for all supported versions without having to install all those python versions, but then again I'm pretty happy if we get proper cross compilation support for installed interpreters.

[mtp401]

@konstin this should be good to go. Tested fine on my end.

[konstin]

Thank you!