macOS Cross-Compile `release_29`: `ffs::ffs` missing in `adios2-config.cmake` #3716

ax3l · 2023-07-26T07:12:23Z

I am trying to build release_29 due to the many patches that are needed to get our all-static dependency wheels to build.

I am currently blocked by:

...
    -- Found ADIOS2: /usr/local/lib/cmake/adios2/adios2-config.cmake (found suitable version "2.9.0", minimum required is "2.7.0") found components: C CXX
    CMake Error at CMakeLists.txt:444 (find_package):
      Found package configuration file:
  
        /usr/local/lib/cmake/adios2/adios2-config.cmake
  
      but it set ADIOS2_FOUND to FALSE so package "ADIOS2" is considered to be
      NOT FOUND.  Reason given by package:
  
      The following imported targets are referenced, but are missing: ffs::ffs

openPMD/openPMD-api#1387

when cross-compiling for macOS arm64 (from x86).

cc @vicentebolea @eisenhauer

Note that ADIOS2 itself builds & installs fine, it is the installed adios2-config.cmake package that seems to forget to define one of the used targets.

The text was updated successfully, but these errors were encountered:

vicentebolea · 2023-07-26T15:39:32Z

@ax3l thanks for reporting. Can i get the full configure/build output.

ax3l · 2023-07-26T17:31:19Z

@vicentebolea
Yes, let me download and link the details from the cibuildwheels setup in openPMD/openPMD-api#1387. library_builders.sh has the build logic in this link.

Build is for:

arm64 wheel on macos-11
host system is macos-11 on x86-64

Complete logs are: macos-xcompile.zip

vicentebolea · 2023-07-26T23:56:47Z

I see is it possible to configure openpmd with --trace-expand and write the the output. I have seen this error before and as I recall it was something to do with ADIOS2_DIR must be specified as a env variable rather than a cmake flag.

ax3l · 2023-07-27T00:39:56Z

Thanks, ok let me see how I can pipe that eloquently through cibuildwheels through pip through setup.py back into CMake 😅
--trace-expand has no environment variable equivalent, does it?

ax3l · 2023-07-27T06:35:04Z

@vicentebolea ok, got it here:
cmake-trace.zip

And /usr/local/lib/cmake/adios2/adios2-config.cmake seems to contain something like:

2023-07-27T01:14:40.8017740Z /Users/runner/work/openPMD-api/openPMD-api/src
2023-07-27T01:14:40.8018560Z cmake_minimum_required(VERSION 3.1)
2023-07-27T01:14:40.8019170Z 
2023-07-27T01:14:40.8019800Z set(_ADIOS2_CMAKE_MODULE_PATH ${CMAKE_MODULE_PATH})
2023-07-27T01:14:40.8020670Z list(INSERT CMAKE_MODULE_PATH 0 "${CMAKE_CURRENT_LIST_DIR}")
2023-07-27T01:14:40.8021290Z 
2023-07-27T01:14:40.8022750Z if(NOT OFF)
2023-07-27T01:14:40.8024300Z   if(NOT )
2023-07-27T01:14:40.8025360Z     set(EVPath_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
2023-07-27T01:14:40.8026110Z   endif()
2023-07-27T01:14:40.8026650Z 
2023-07-27T01:14:40.8027120Z   if(NOT OFF)
2023-07-27T01:14:40.8027850Z     set(atl_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
2023-07-27T01:14:40.8028530Z   endif()
2023-07-27T01:14:40.8029070Z 
2023-07-27T01:14:40.8029540Z   if(NOT OFF)
2023-07-27T01:14:40.8030260Z     set(dill_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
2023-07-27T01:14:40.8030960Z   endif()
2023-07-27T01:14:40.8031500Z 
2023-07-27T01:14:40.8031940Z   if(NOT OFF)
2023-07-27T01:14:40.8032670Z     set(ffs_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
2023-07-27T01:14:40.8033360Z   endif()
2023-07-27T01:14:40.8033890Z 
2023-07-27T01:14:40.8034380Z   if(NOT OFF)
2023-07-27T01:14:40.8035070Z     set(enet_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
2023-07-27T01:14:40.8035870Z   endif()
2023-07-27T01:14:40.8036550Z endif()
2023-07-27T01:14:40.8037100Z 
2023-07-27T01:14:40.8037670Z set(${CMAKE_FIND_PACKAGE_NAME}_CONFIG "${CMAKE_CURRENT_LIST_FILE}")
2023-07-27T01:14:40.8038690Z include("${CMAKE_CURRENT_LIST_DIR}/adios2-config-common.cmake")
2023-07-27T01:14:40.8039380Z 
2023-07-27T01:14:40.8039920Z set(CMAKE_MODULE_PATH ${_ADIOS2_CMAKE_MODULE_PATH})
2023-07-27T01:14:40.8040800Z unset(_ADIOS2_CMAKE_MODULE_PATH)

ax3l · 2023-07-27T06:38:56Z

Side question: FFS is a mandatory ADIOS2 dependency or can it be disabled? (I did not find an option for it.)

eisenhauer · 2023-07-27T10:20:47Z

Side question: FFS is a mandatory ADIOS2 dependency or can it be disabled? (I did not find an option for it.)

The options to disable it are indirect. It is required both for SST and for BP5. If you disable both of those FFS will not be built. (But BP5 is now the default file format. If you disable it there are some tests that won't pass, functionality will be lost, etc.)

ax3l · 2023-07-27T16:13:45Z

Thanks Greg, yes that makes sense then to not ship without it.

@eisenhauer @vicentebolea does the detailed CMake traceback above provide you with the info we need?
I also checked again in conda-forge/adios2-feedstock#74 and saw not FFS specific work-around there.

vicentebolea · 2023-07-27T20:34:59Z

I have found the culprit but not the reason of this happening.

Looking at the trace, /usr/local/lib/cmake/adios2/adios2-config.cmake(7) should be if (NOT OFF) since the OFF is ADIOS2_USE_EXTERNAL_EVPATH, however it is for some reason undef.


2023-07-27T01:15:15.9040940Z 
2023-07-27T01:15:15.9041520Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(3):  set(_ADIOS2_CMAKE_MODULE_PATH /Users/runner/work/openPMD-api/openPMD-api/src/share/openPMD/cmake )
2023-07-27T01:15:15.9042260Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(4):  list(INSERT CMAKE_MODULE_PATH 0 /usr/local/lib/cmake/adios2 )
2023-07-27T01:15:15.9042820Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(6):  if(NOT OFF )
2023-07-27T01:15:15.9043330Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(7):  if(NOT )
2023-07-27T01:15:15.9043840Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(11):  if(NOT OFF )
2023-07-27T01:15:15.9044440Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(12):  set(atl_DIR /usr/local/lib/cmake/adios2/thirdparty )
2023-07-27T01:15:15.9045010Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(15):  if(NOT OFF )
2023-07-27T01:15:15.9045620Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(16):  set(dill_DIR /usr/local/lib/cmake/adios2/thirdparty )
2023-07-27T01:15:15.9046510Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(19):  if(NOT OFF )
2023-07-27T01:15:15.9047120Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(20):  set(ffs_DIR /usr/local/lib/cmake/adios2/thirdparty )
2023-07-27T01:15:15.9047680Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(23):  if(NOT OFF )
2023-07-27T01:15:15.9048280Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(24):  set(enet_DIR /usr/local/lib/cmake/adios2/thirdparty )
2023-07-27T01:15:15.9048960Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(28):  set(ADIOS2_CONFIG /usr/local/lib/cmake/adios2/adios2-config.cmake )
2023-07-27T01:15:15.9049670Z   /usr/local/lib/cmake/adios2/adios2-config.cmake(29):  include(/usr/local/lib/cmake/adios2/adios2-config-common.cmake )

Here is the source file of adios2-config.cmake:

if(NOT @BUILD_SHARED_LIBS@)
  if(NOT @ADIOS2_USE_EXTERNAL_EVPATH@)
    set(EVPath_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
  endif()

  if(NOT @ADIOS2_USE_EXTERNAL_ATL@)
    set(atl_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
  endif()

  if(NOT @ADIOS2_USE_EXTERNAL_DILL@)
    set(dill_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
  endif()

  if(NOT @ADIOS2_USE_EXTERNAL_FFS@)
    set(ffs_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
  endif()

  if(NOT @ADIOS2_USE_EXTERNAL_ENET@)
    set(enet_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty)
  endif()
endif()

It is located at cmake/adios2-config-install.cmake.in. we need set(EVPath_DIR ${CMAKE_CURRENT_LIST_DIR}/thirdparty) since later it will find_dependency(EVPATH) which will load all the other (ffs, atl, dill, enet) thirdparty cmake config files

Note that this is only happening with static build

vicentebolea · 2023-07-27T20:36:02Z

@ax3l can you also run the --trace-expand for the configuration of Adios2, this might bring some light on why this is happening

ax3l · 2023-07-28T00:40:41Z

@vicentebolea no problem, here is the log output, including --trace-expand for the ADIOS2 build step.
cmake-adios2-trace.zip

ax3l · 2023-07-28T00:43:16Z

@vicentebolea I think the part this script above is not expecting is in

if(NOT @BUILD_SHARED_LIBS@)

I do build all-static for wheels, so the most outer if becomes

if(NOT FALSE)

ax3l · 2023-07-28T18:44:55Z

@vicentebolea is the if(NOT @BUILD_SHARED_LIBS@) generally the issue here? Don't we need to define the CMake targets either way, independent if we use them as shared or static targets?

vicentebolea · 2023-07-28T19:50:35Z

The issue happens because EVPATH dependency gets properly exported when using SST, however, even without SST we use ffs which gets exposed through EVPATH. @ax3l the workaround should be to enable SST or disable BP5.

Things to do:

Properly export ffs in adios2-config when SST is disabled.

ax3l · 2023-07-28T21:20:54Z

Ah, but I need BP5 and was bit able to cross-compile SST in the past (#3116 - fixed in 2.8.0+).
Let me try to turn SST on again for ADIOS 2.9.0.

Update: that works for now 🎉
I just need to disable SST here and there for some platforms, so #3724 will be appreciated 🙏

vicentebolea · 2023-08-23T23:14:44Z

I believe that this has been resolved in #3760

vicentebolea · 2024-07-19T22:15:29Z

Closing this since we cannot reproduce this anymore. Please re-open if you observe this again

ax3l changed the title ~~release_29: ffs not found~~ release_29: ffs::ffs not found macOS Cross-Compile Jul 26, 2023

ax3l changed the title ~~release_29: ffs::ffs not found macOS Cross-Compile~~ macOS Cross-Compile release_29: ffs::ffs missing in adios2-config.cmake Jul 26, 2023

vicentebolea self-assigned this Jul 26, 2023

vicentebolea mentioned this issue Jul 28, 2023

Properly export ffs in adios2-config when SST is disabled. #3724

Closed

1 task

ax3l mentioned this issue Feb 5, 2024

CI: All-Static builds (Linux/macOS/Win) #4025

Closed

vicentebolea closed this as completed Jul 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macOS Cross-Compile `release_29`: `ffs::ffs` missing in `adios2-config.cmake` #3716

macOS Cross-Compile `release_29`: `ffs::ffs` missing in `adios2-config.cmake` #3716

ax3l commented Jul 26, 2023 •

edited

Loading

vicentebolea commented Jul 26, 2023

ax3l commented Jul 26, 2023

vicentebolea commented Jul 26, 2023

ax3l commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

eisenhauer commented Jul 27, 2023

ax3l commented Jul 27, 2023 •

edited

Loading

vicentebolea commented Jul 27, 2023 •

edited

Loading

vicentebolea commented Jul 27, 2023

ax3l commented Jul 28, 2023

ax3l commented Jul 28, 2023 •

edited

Loading

ax3l commented Jul 28, 2023 •

edited

Loading

vicentebolea commented Jul 28, 2023

ax3l commented Jul 28, 2023 •

edited

Loading

vicentebolea commented Aug 23, 2023

vicentebolea commented Jul 19, 2024

macOS Cross-Compile release_29: ffs::ffs missing in adios2-config.cmake #3716

macOS Cross-Compile release_29: ffs::ffs missing in adios2-config.cmake #3716

Comments

ax3l commented Jul 26, 2023 • edited Loading

vicentebolea commented Jul 26, 2023

ax3l commented Jul 26, 2023

vicentebolea commented Jul 26, 2023

ax3l commented Jul 27, 2023 • edited Loading

ax3l commented Jul 27, 2023 • edited Loading

ax3l commented Jul 27, 2023 • edited Loading

eisenhauer commented Jul 27, 2023

ax3l commented Jul 27, 2023 • edited Loading

vicentebolea commented Jul 27, 2023 • edited Loading

vicentebolea commented Jul 27, 2023

ax3l commented Jul 28, 2023

ax3l commented Jul 28, 2023 • edited Loading

ax3l commented Jul 28, 2023 • edited Loading

vicentebolea commented Jul 28, 2023

ax3l commented Jul 28, 2023 • edited Loading

vicentebolea commented Aug 23, 2023

vicentebolea commented Jul 19, 2024

macOS Cross-Compile `release_29`: `ffs::ffs` missing in `adios2-config.cmake` #3716

macOS Cross-Compile `release_29`: `ffs::ffs` missing in `adios2-config.cmake` #3716

ax3l commented Jul 26, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 27, 2023 •

edited

Loading

vicentebolea commented Jul 27, 2023 •

edited

Loading

ax3l commented Jul 28, 2023 •

edited

Loading

ax3l commented Jul 28, 2023 •

edited

Loading

ax3l commented Jul 28, 2023 •

edited

Loading