-
-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cross-compilation support #1203
Comments
any status or thoughts? |
Thanks for the update, looks quite promising and glad to see progress |
Any updates on this? @gnuoyd I noticed both PRs have now been merged: what other pieces need to get done? |
#1410 also is related and I will document the CMake process there and then create a cross-compile document in the release_docs folder. |
@byrnHDF trying to build eac2cd5 (most recent commit in
For reference, CMake invocation: cmake .. -DCMAKE_INSTALL_PREFIX=${prefix} \
-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TARGET_TOOLCHAIN} \
-DHDF5_BUILD_CPP_LIB=OFF \
-DONLY_SHARED_LIBS=ON \
-DHDF5_BUILD_HL_LIB=ON \
-DHDF5_ENABLE_Z_LIB_SUPPORT=ON \
-DHDF5_ENABLE_SZIP_SUPPORT=OFF \
-DHDF5_ENABLE_SZIP_ENCODING=OFF \
-DBUILD_TESTING=OFF Am I missing something? |
The next step is to set those try-run values in the your preset H5Tinit.c file. I think H5Tinit.c is created by the above. See src/CMakeLists.txt about line 1035. References: |
I thought the entire point of supporting cross-compilation was to not have to manually set the output of try-run values for all possible target platforms. |
I think we just need the CMakeCache.txt files, no? https://cmake.org/cmake/help/latest/command/try_run.html#behavior-when-cross-compiling |
Here's my src/build-MINGW64/CMakeCache.txt from a MINGW64 run:
|
Sounds like we need to copy the values into the generated TryRunResults.cmake and then pass those settings into |
Our latest develop has no errors compiling with mingw64 on an ubuntu 22.04 system. |
Is your mingw64 on a windows machine? I haven't got one of those workers available. |
Yes, I'm running mingw64 under msys2 on windows. |
Maybe what is needed is some more preset values in Configure for mingw on windows environments? |
long double has always been trouble in cygwin, mingw on windows and such. dtarith test usually fails on those as well. I think it has to do with compiler tricks. (Not official statement, just a personal opinion). |
ConversionTests.c file is an interesting read. |
We may be getting a bit distracted by the mingw-w64 aspect. As far as I can tell we have a solution to build on that platform. That is we have a functional native build solution for msys2/mingw-w64. What I would like to do is send @giordano a file, my CMakeCache.txt for example, so that we can also build mingw-w64 binaries from Linux via cross compilation. Cmake seems to have some very rudimentary infrastructure to support this. Basically, if we can transmit information The process seems to be as follows:
|
TryRun isn't a great way to do cross-compilation though. Quite the opposite in fact. What stevengj/hdf5@6b05455 showed is that most of those checks are actually compiler checks which don't need to run code in the first place. |
Yes that is basically the process. |
If we wanted to bypass try_run and try_compile, we could use Per @derobins , The HDF Group is leaning towards retiring autotools in favor of cmake. I've commented there that an issue with this is it closing off Steven G. Johnson's route of populating this. Perhaps the "emulation" route might be a way to salvage this? |
Eliminating tryrun has been discussed and is preferred. It is the details in doing it for all the platforms where hdf is used. |
Are you suggesting that we can the toolchain files located at the link below to include type information that https://github.com/HDFGroup/hdf5/tree/develop/config/toolchain |
Maybe, I think it would be a good place for specific settings that are specific to that build env. |
Actually, a toolchain could be gotten from a URL using the FetchContent (CMake has an example) |
The Java CPP project has cross compiling setup: https://github.com/bytedeco/javacpp-presets/tree/master/hdf5 |
Hello, has this problem been solved |
I set these variables to OFF, and cmake succeeds. But there was a tricky problem when make: Generated files H5detect and H5make_libsettings, and report an error "H5make_libsettings: cannot execute binary file"; |
As far as I know this has not been done. Preferably, the need to run an executable on the target platform could be replaced by some static assertions in C and Fortran so that the settings could be detected by compilation alone. https://en.cppreference.com/w/c/language/_Static_assert https://fortran-lang.discourse.group/t/static-assert-in-fortran/5687 |
H5make_libsettings and H5detect is used to generate some files, I am not sure what you mean. |
As you noticed, these programs need to be run on the target platform. To make this work you need to compile the program, run it on the target platform, and then transfer some files back to the host compilation environment. You can see an example of this being done for the Julia ecosystem here: Rather than detecting floating point properties at runtime as in H5detect.c, we might be able to detect some of these properties at compile time. For example, if one were using GCC one may be able to detect if we are using IEEE 754 compliant floating point by checking some macro values. #include<assert.h>
#include<float.h>
int main(void) {
static_assert(sizeof(float) == 4);
static_assert(FLT_RADIX == 2);
static_assert(FLT_MANT_DIG == 24);
static_assert(FLT_DIG == 6);
static_assert(FLT_MIN_EXP == -125);
static_assert(FLT_MIN_10_EXP == -37);
static_assert(FLT_MAX_EXP == 128);
static_assert(FLT_MAX_10_EXP == +38);
static_assert(FLT_MIN == 1.17549435E-38F);
static_assert(FLT_MAX == 3.40282347E+38F);
static_assert(FLT_EPSILON == 1.19209290E-07F);
static_assert(sizeof(double) == 8);
static_assert(DBL_MANT_DIG == 53);
static_assert(DBL_DIG == 15);
static_assert(DBL_MIN_EXP == -1021);
static_assert(DBL_MIN_10_EXP == -307);
static_assert(DBL_MAX_EXP == 1024);
static_assert(DBL_MAX_10_EXP == 308);
static_assert(DBL_MAX == 1.7976931348623157E+308);
static_assert(DBL_MIN == 2.2250738585072014E-308);
static_assert(DBL_EPSILON == 2.2204460492503131E-016);
} |
We can make the existing cmake configure process simpler, caching the values of the try-run, see https://cmake.org/cmake/help/latest/command/try_run.html __TRYRUN_OUTPUT ` |
You could eliminate the need for most of H5detect.c by checking if these compile. float_assert.cpp:
long_double_assert.cpp (might not work on 32-bit systems)
For 32-bit systems not using MSVC:
For Microsoft Visual Studio compiler (MSVC):
|
H5detect is designed to allow for cross-compilation (also) |
Yes, the problem of cross-compilation isn't caching the result of try_run, but the try_run itself! And most of the try_run checks can be replaced with compiler checks, as suggested by Mark, and demonstrated in the now-5-year-old fork https://github.com/stevengj/hdf5 linked in the original post of this issue |
Say we're trying to cross-compile everything on x86_64 based cloud instance host for continuous integration and we do not have access to the target hardware or an emulator for that environment. How many platforms could we accurately populate H5init.c with via C preprocessor macros alone given gcc or clang based toolchains? |
To check this, we could use Godbolt to see how many toolchains are able to compile the static assertions. |
Can this issue be closed?
Cross compiling should basically just work now |
Yep should be fixed now |
In Julia this is the relevant PR that made the changes to enable cross-compilation JuliaPackaging/Yggdrasil#6551 |
@giordano should a new issue to update the script in Yggrasil be opened instead? |
Lack of cross-compilation from linux to other platforms would significantly help distribution of binaries in the Julia ecosystem. This has been a long-standing issue. See e.g. JuliaPackaging/Yggdrasil#567, https://forum.hdfgroup.org/t/cross-compiling-for-windows/6735, to mention two threads. There have been several attempts at it including the fork https://github.com/stevengj/hdf5 and in addition Steven Varga has worked on looking at this in the past.
The text was updated successfully, but these errors were encountered: