#9303 - Effectively static link fortran utilities (and make them work on M1) by jmarrec · Pull Request #9676 · NatLabRockies/EnergyPlus

jmarrec · 2022-09-27T14:35:17Z

Pull request overview

Fixes Fortran utilities do not work in mac M1 (arm64) package when installed #9303
This removes any doubt I had regarding the fact that the 22.2.0-RC2 did not contain libgcc_s.1.dylib until v22.2.0-IOFreeze
Tested at https://github.com/jmarrec/EnergyPlus/releases/tag/9303_Fortran_UtilitiesLinking and locally on an M1 mac (LOTS of warning suppressed, and fortran utilities work)

Pull Request Author

Add to this list or remove from it as applicable. This is a simple templated set of guidelines.

Reviewer

This will not be exhaustively relevant to every PR.

…ll statically link stuff

…1 doesn't exist yet ``` [24504] Operation \"Execute\" with arguments \"ln; -sf; @TargetDir@/energyplus.1; /usr/local/share/man/man1; UNDOEXECUTE; rm; /usr/local/share/man/man1/energyplus.1\" failed: Execution failed (Unexpected exit code: 1): \"ln -sf /Applications/EnergyPlus-22-2-0/energyplus.1 /usr/local/share/man/man1\" [24504] installationErrorWithCancel : Installer Error : Error during installation process (Symlinks): Execution failed (Unexpected exit code: 1): "ln -sf /Applications/EnergyPlus-22-2-0/energyplus.1 /usr/local/share/man/man1" Retry|Ignore|Cancel ```

jmarrec · 2022-09-27T14:40:48Z

cmake/Fortran.cmake

+    # `-static` isn't possible on mac. From gcc man page:
+    # > This option will not work on Mac OS X unless all libraries (including libgcc.a) have also been compiled with -static.
+    # > Since neither a static version of libSystem.dylib nor crt0.o are provided, this option is not useful to most people.
+
+    # master of gcc has a fix via a new `-static-libquadmath` but it's not even in gcc 12.2.0. So instead we have to do all kinds of shenanigans
+
+    enable_language(Fortran)  # Needed to populate ${CMAKE_Fortran_IMPLICIT_LINK_DIRECTORIES}
+    # Debug:
+    # set(Fortran_VERBOSE "-v")
+    # Find the static libquadmath maually
+    find_library(static-libquadmath NAMES quadmath.a libquadmath.a REQUIRED PATHS ${CMAKE_Fortran_IMPLICIT_LINK_DIRECTORIES})
+    message(STATUS "Found static-libquadmath at ${static-libquadmath} by searching in CMAKE_Fortran_IMPLICIT_LINK_DIRECTORIES=${CMAKE_Fortran_IMPLICIT_LINK_DIRECTORIES}")
+    target_compile_options(fortran_project_options INTERFACE
+      ${Fortran_VERBOSE}
+    )
+    target_link_options(fortran_project_options INTERFACE
+      ${Fortran_VERBOSE}
+      -nodefaultlibs
+      -static-libgfortran
+      -lgfortran
+      -static-libgcc
+      -lgcc
+      -lemutls_w
+      ${static-libquadmath}
+      -lSystem
+    )
+
+    message(DEBUG "CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES-${CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES}")
+    #    => CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES-gfortran;emutls_w;gcc;quadmath;emutls_w;gcc;gcc
+    list(REMOVE_DUPLICATES CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES)
+    list(TRANSFORM CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES REPLACE "quadmath" " ${static-libquadmath}")
+    message(DEBUG "CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES-${CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES}")
+
+    # We unset this so that CMake doesn't try to read -lquadmath etc
+    unset(CMAKE_Fortran_IMPLICIT_LINK_LIBRARIES)
+
+    # We could technically just set it to TRUE here. But it shouldn't do anything anyways, and it's a fallback
+    # set(FORTRAN_STATIC_EXE TRUE)


That's it. Obviously the pain point was clearly getting to a working implementation. I had to inspect the source code of both CMake and GCC.

I tried passing a custom -specs=${CMAKE_CURRENT_LIST_DIR}/libgfortran.spec with this content

%rename lib liborig2 *lib: %{!static-libgfortran: -lquadmath } %{static-libgfortran: libquadmath.a%s} %(libgcc) %(liborig2)

but that conflicted with the base spec somehow (/opt/homebrew//Cellar/gcc/12.2.0/lib/gcc/current/libgfortran.spec). In the system one if I make the same change it worked fine though, but that's not portable to another machine...

%rename lib liborig -*lib: -lquadmath %(libgcc) %(liborig) +*lib: %{!static-libgfortran: -lquadmath } %{static-libgfortran: libquadmath.a%s} %(libgcc) %(liborig)

Note that in the future, GCC will have a -static-libquadmath option which should remove of this workaround. It's not in 12.2.0 though, only on master: gcc-mirror/gcc@745be54

install_name_tool isn't called anymore, so there's no code signature invalidation, and therefore the utility runs on M1 (which has hardened runtime).

My initial try was to adjust fixup_executable in ProjectMacros.cmake (try to call codesign -s -f -- <mylib>) and try to pass RUNTIME_DEPENDENCIES to the individual install(TARGET) in the fortran subdirectory (eg https://github.com/NREL/EnergyPlus/blob/1d22b9a61f7cd00b9e7b0ef27e4b24ffb6b849f3/src/ParametricPreprocessor/CMakeLists.txt#L26) (which requires cmake 3.21 though), but it wasn't working cleanly and I realized I could potentially make (almost-)static linking work.

(Note: I say almost-static because technically this isn't a statically linked lib, but it doesn't depends on anything else than libSystemB.dylib)

jmarrec · 2022-09-27T14:45:44Z

cmake/Install.cmake


  # You need at least one "install(..." command for it to be registered as a component
  install(CODE "MESSAGE(\"Creating symlinks.\")" COMPONENT Symlinks)
+  install(FILES "${PROJECT_SOURCE_DIR}/doc/man/energyplus.1" DESTINATION "./" COMPONENT Symlinks)


Install man page on mac

jmarrec · 2022-09-27T14:46:34Z

cmake/qtifw/install_unix_createsymlinks.qs

+      component.addElevatedOperation("Execute", "mkdir", "-p", linktarget);
+
+      component.addElevatedOperation("Execute", "ln", "-sf", "@TargetDir@/energyplus.1", linktarget + "/energyplus.1",


Install man page on mac (and linux): create the /usr/local/share/man/man1 directory if it doesn't exist (it doesn't by default) to you avoid this annoying warning

[24504] Operation \"Execute\" with arguments \"ln; -sf; @TargetDir@/energyplus.1; /usr/local/share/man/man1; UNDOEXECUTE; rm; /usr/local/share/man/man1/energyplus.1\" failed: Execution failed (Unexpected exit code: 1): \"ln -sf /Applications/EnergyPlus-22-2-0/energyplus.1 /usr/local/share/man/man1\" [24504] installationErrorWithCancel : Installer Error : Error during installation process (Symlinks): Execution failed (Unexpected exit code: 1): "ln -sf /Applications/EnergyPlus-22-2-0/energyplus.1 /usr/local/share/man/man1" Retry|Ignore|Cancel

Myoldmopar

Wow, this is some effort @jmarrec. I think I will tag RC3 after all and give this a quick test on my machine. I realize my machine isn't a clean VM, but if you are seeing success on your M1, and I'm seeing success on my machine, I'd feel pretty good about it.

…tiesLinking

mjwitte · 2022-09-27T15:05:52Z

@jmarrec Do these changes target only M1 Mac, or does it touch other platforms as well?

jmarrec · 2022-09-27T19:56:35Z

@jmarrec Do these changes target only M1 Mac, or does it touch other platforms as well?

Currently it tries to do an effective static linking of fortran libs on macOS (any arch) with Gfortran (not ifort).

Myoldmopar · 2022-09-28T02:25:20Z

OK, that was some fun detective work. Great job @jmarrec and there are surely some things we can continue to clean up in here when time is better. Thanks!

cf NatLabRockies/EnergyPlus#9676

#9676 Follow Up - Make it more resilient to subtle differences in gfortran versions

jmarrec added 2 commits September 27, 2022 14:43

On macOS where gfortran can't use --static, figure out a way to sti…

781b81e

…ll statically link stuff

jmarrec requested a review from Myoldmopar September 27, 2022 14:35

jmarrec self-assigned this Sep 27, 2022

jmarrec changed the base branch from develop to PrepFor222 September 27, 2022 14:35

jmarrec commented Sep 27, 2022

View reviewed changes

Myoldmopar approved these changes Sep 27, 2022

View reviewed changes

Base automatically changed from PrepFor222 to develop September 27, 2022 15:04

Merge remote-tracking branch 'origin/develop' into 9303_Fortran_Utili…

860fd63

…tiesLinking

jmarrec force-pushed the 9303_Fortran_UtilitiesLinking branch from 3e02160 to 860fd63 Compare September 27, 2022 19:37

Myoldmopar added Defect Includes code to repair a defect in EnergyPlus Developer Issue Related to cmake, packaging, installers, or developer tooling (CI, etc) labels Sep 27, 2022

Myoldmopar merged commit c249759 into develop Sep 28, 2022

Myoldmopar deleted the 9303_Fortran_UtilitiesLinking branch September 28, 2022 02:25

jmarrec added a commit to NatLabRockies/OpenStudio that referenced this pull request Sep 28, 2022

Update hash for E+ arm64 + adjust fortran libs

9c46ec2

cf NatLabRockies/EnergyPlus#9676

jmarrec mentioned this pull request Sep 29, 2022

#9676 Follow Up - Make it more resilient to subtle differences in gfortran versions #9682

Merged

20 tasks

Myoldmopar added a commit that referenced this pull request Sep 30, 2022

Merge pull request #9682 from jmarrec/9303_Fortran_UtilitiesLinking

efba7af

#9676 Follow Up - Make it more resilient to subtle differences in gfortran versions

jmarrec mentioned this pull request Jan 18, 2023

Slab pre-processor not working on macOS #9197

Closed

perazz mentioned this pull request Mar 30, 2023

fix macOS CI for fpm-bootstrap executables built with gcc-9 fortran-lang/fpm#861

Merged

jmarrec mentioned this pull request Sep 9, 2023

mac zsh kills ReadVarsESO #10130

Closed

This was referenced Nov 20, 2023

EnergyPlus ExpandObjects program will not launch NatLabRockies/OpenStudio#5050

Closed

libgcc_s.1.1.dylib dependency is back on mac x86_64 #10306

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

#9303 - Effectively static link fortran utilities (and make them work on M1)#9676

#9303 - Effectively static link fortran utilities (and make them work on M1)#9676
Myoldmopar merged 3 commits intodevelopfrom
9303_Fortran_UtilitiesLinking

jmarrec commented Sep 27, 2022

Uh oh!

jmarrec Sep 27, 2022

Uh oh!

jmarrec Sep 27, 2022

Uh oh!

jmarrec Sep 27, 2022

Uh oh!

jmarrec Sep 27, 2022

Uh oh!

Myoldmopar left a comment

Uh oh!

mjwitte commented Sep 27, 2022

Uh oh!

jmarrec commented Sep 27, 2022

Uh oh!

Myoldmopar commented Sep 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

		component.addElevatedOperation("Execute", "mkdir", "-p", linktarget);

		component.addElevatedOperation("Execute", "ln", "-sf", "@TargetDir@/energyplus.1", linktarget + "/energyplus.1",

Comments

Conversation

jmarrec commented Sep 27, 2022

Pull request overview

Pull Request Author

Reviewer

Uh oh!

jmarrec Sep 27, 2022

Choose a reason for hiding this comment

Uh oh!

jmarrec Sep 27, 2022

Choose a reason for hiding this comment

Uh oh!

jmarrec Sep 27, 2022

Choose a reason for hiding this comment

Uh oh!

jmarrec Sep 27, 2022

Choose a reason for hiding this comment

Uh oh!

Myoldmopar left a comment

Choose a reason for hiding this comment

Uh oh!

mjwitte commented Sep 27, 2022

Uh oh!

jmarrec commented Sep 27, 2022

Uh oh!

Myoldmopar commented Sep 28, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants