NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

mathomp4 · 2024-03-18T14:27:12Z

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

I'm testing Open MPI 5.0.2 (and 4.1.6)

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Both installed from tarfile.

Configure command is:

$ mkdir build-nag-7.2.01 && cd build-nag-7.2.01
$ ../configure --disable-wrapper-rpath --disable-wrapper-runpath \
      CC=gcc CXX=g++ FC=nagfor \
      --with-pmix=internal --with-hwloc=internal --with-libevent=internal \
      --prefix=/ford1/share/gmao_SIteam/MPI/openmpi/X.Y.Z/nag-7.2.01 |& tee configure.nag-7.2.01.log

where X.Y.Z is either 4.1.6 or 5.0.2

Please describe the system on which you are running

Operating system/version: RHEL8
Computer hardware: Intel(R) Xeon(R) CPU E5-2680 v4
Network type: Ethernet (I think? Single node)

Details of the problem

I am trying to build Open MPI 5.0.2 with NAG Fortran 7.2.01 (with GCC 12.1 as C/C++) and all works until:

make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
  CCLD     libmpi_mpifh.la
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information

if I build with V=1:

make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/n
ag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/libopen-pal.la pr
ofile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la
 /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/libpmix.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs/libopen-pal.so -lpthread -lrt /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs/libpmix.so -lutil /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2937: libmpi_mpifh.la] Error 1

So, I thought, let me try Open MPI 4.1.6 and:

...
make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
  GENERATE sizeof_f.f90
  FC       sizeof_f.lo
NAG Fortran Compiler Release 7.2(Shin-Urayasu) Build 7201
[NAG Fortran Compiler normal termination]
  FCLD     libmpi_mpifh_sizeof.la
  CCLD     libmpi_mpifh.la
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
Making all in mpi/fortran/use-mpi-tkr
make[2]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/use-mpi-tkr'
  GENERATE mpi-tkr-sizeof.h
  GENERATE mpi-tkr-sizeof.f90
  FC       mpi_aint_add_f90.lo
  FC       mpi_aint_diff_f90.lo
  FC       mpi_comm_spawn_multiple_f90.lo
...

It builds successfully.

I looked around here and I did see #4032 which is old, but, well, NAG is there, pthread is there... Maybe some old fix for NAG and Open MPI 2 got into 3, stayed around for 4, but got lost with 5?

NOTE: I was going to try and maybe see if I could use our new NAG setup to help with #9795, but that is a configure time fail so...

The text was updated successfully, but these errors were encountered:

mathomp4 · 2024-03-18T14:28:35Z

Oh, and I can say I've built Open MPI 5.0.2 with NAG 7.2 (and Apple Clang) on my macOS laptop, so I don't think is nagfor per se. Maybe something Clang is happy with but GCC isn't? 🤷🏼

bosilca · 2024-03-18T14:35:12Z

gcc -dumpspecs | grep pthread and look for the lines starting with pthread. On my OSX with gcc (12.3.0 installed via macports) I see that -pthread will be translated at compile time into -D_REENTRANT

mathomp4 · 2024-03-18T14:42:05Z

@bosilca Well, on my mac, I build Open MPI for NAG with clang as the C compiler, not GCC:

❯ clang -dumpspecs | grep pthread
clang: error: unsupported option '-dumpspecs'
clang: error: no input files

But, on the RHEL 8 machine I see:

@$ gcc -dumpspecs | grep pthread
%{posix:-D_POSIX_SOURCE} %{pthread:-D_REENTRANT}
...

Of course, the error seems to say -pthread is not a flag ld knows (maybe via -Wl,-pthread from my verbose make) and:

@$ ld -pthread
ld: unrecognized option '-pthread'
ld: use the --help option for usage information

The verbose make says:

-ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0

It's like configure has decided to pass -pthread to ld but that's not a flag ld knows?

ggouaillardet · 2024-03-21T06:54:37Z

When nagfor is used, -pthread (that can be pulled from a .la file) is replaced with -Wl,-pthread via a patch we apply to config/ltmain.sh.

In your case, gcc is passed both -pthread and -Wl,-pthread which means ld will be passed the -pthread flag.

I do not quite see the connection between these ...

ggouaillardet · 2024-03-21T07:07:24Z

since you built 4.1.6, can you please post the output of

cd ompi/mpi/fortran/mpif-h
rm libmpi_mpifh.la
make V=1

mathomp4 · 2024-03-21T11:26:58Z

@ggouaillardet Here is the make output for Open MPI 4.1.6:

Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG -finline-functions -fno-strict-aliasing -mcx16 -pthread -version-info 70:0:30  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/4.1.6/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/pcollreq/mpif-h/libmpiext_pcollreq_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lrt -lm -lutil  -lz 
libtool: link: rm -fr  .libs/libmpi_mpifh.la .libs/libmpi_mpifh.lai .libs/libmpi_mpifh.so .libs/libmpi_mpifh.so.40 .libs/libmpi_mpifh.so.40.30.0 .libs/libmpi_mpifh.so.40.30.0T
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/pcollreq/mpif-h/.libs/libmpiext_pcollreq_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/4.1.6/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/orte/.libs/libopen-rte.so /ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/opal/.libs/libopen-pal.so -ldl -ludev -lpciaccess -lrt -lm -lutil -lz  -O3 -mcx16 -pthread   -pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.30.0
libtool: link: (cd ".libs" && rm -f "libmpi_mpifh.so.40" && ln -s "libmpi_mpifh.so.40.30.0" "libmpi_mpifh.so.40")
libtool: link: (cd ".libs" && rm -f "libmpi_mpifh.so" && ln -s "libmpi_mpifh.so.40.30.0" "libmpi_mpifh.so")
libtool: link: ( cd ".libs" && rm -f "libmpi_mpifh.la" && ln -s "../libmpi_mpifh.la" "libmpi_mpifh.la" )
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-4.1.6/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'

and here is the same for 5.0.2:

Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/libpmix.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2/nag-7.2.01/lib -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -L/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/opal/.libs/libopen-pal.so -lpthread -lrt /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/openpmix/src/.libs/libpmix.so -lutil /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so -lm -ludev -ldl  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:2937: libmpi_mpifh.la] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/openmpi-5.0.2/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make: *** [Makefile:3435: all-recursive] Error 1

Doing a compare I see for 4.1.6:

-O3 -mcx16 -pthread   -pthread -Wl,-soname -Wl,libmpi_mpifh.so.40

and for 5.0.2:

-O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40

So a new -Wl, appeared (along with a lot of extra stuff of course).

ggouaillardet · 2024-03-22T04:05:12Z

Thanks @mathomp4

would you mind giving a try with this 4.1.6 tarball?
https://github.com/ggouaillardet/ompi/releases/download/4.1.6/openmpi-4.1.6.tar.bz2

I regenerated a 4.1.6 tarball with the same autotools used by the v5 branch (they are slightly more recent that the autotools used in the v4.1 branch.

mathomp4 · 2024-03-22T18:12:43Z

@ggouaillardet Well, I'm not sure if it's a good thing or bad thing, but I tried your tarball and it built fine with NAG 7.2.01 on the same system.

ggouaillardet · 2024-03-23T01:38:04Z

I concur, thanks for the test!

ggouaillardet · 2024-03-26T06:18:25Z

I think I am on to something, but I do not quite see a good fix for that...

First the issue can be reproduced by

configure FC=nagfor && make
where nagfor can be a symbolic link to a working Fortran compiler (e.g. `gfortran)

with v4.1, from ompi/mpi/fortran/mpif-h/profile/Makefile:

LIBS = -lrt -lm -lutil  -lz  -lhwloc  -levent_core -levent_pthreads

but from v5.x, from `ompi/mpi/fortran/mpif-h/profile/Makefile:

LIBS = -lpthread -lrt -lm -lutil -levent_core -levent_pthreads /home/users/u0001043/build/openmpi-5.0.2-nagfor/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la /home/users/u0001043/build/openmpi-5.0.2-nagfor/3rd-party/openpmix/src/libpmix.la

the major difference is v5 explicitly links with libpmix.la.
Long story short, libpmix.la depends on libhwloc.la and -Wl,-pthread gets pulled into the generated libraries, that will cause some issue later.

libpmix.la nor libhwloc.la are not needed here, so not using them might do the trick.

@jsquyres @bwbarrett any ideas on how to best move forward?

mathomp4 · 2024-03-26T16:49:10Z

This might be the first ever NAG bug I've had that doesn't require purchasing NAG! 😄

Thanks for working on this, @ggouaillardet

ggouaillardet · 2024-04-01T02:16:49Z

@mathomp4 can you please give this commit a try?
ggouaillardet@8d331be

you will need to apply it on a v5 source tree and they run autogen.pl (or autogen.pl --force if applying on an official tarball)

Or you might simply download this already bootstrapped tarball at https://github.com/ggouaillardet/ompi/releases/download/nag_pmix/openmpi-gitclone.tar.bz2 and simply run configure ... && make

mathomp4 · 2024-04-01T13:42:31Z

@ggouaillardet I grabbed the tarball but:

...
NAG Fortran Compiler Release 7.2(Shin-Urayasu) Build 7201
[NAG Fortran Compiler normal termination]
  FCLD     libmpi_mpifh_sizeof.la
  CCLD     libmpi_mpifh.la
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: *** [Makefile:3436: all-recursive] Error 1
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[1]: *** [Makefile:2801: all-recursive] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi'
make: *** [Makefile:1534: all-recursive] Error 1

Should I try the source-and-autogen instead?

ggouaillardet · 2024-04-01T14:46:53Z

I am puzzled since I could pass this with my nagfor symlink...

could you please make V=1 and share the output after CCLD libmpi_mpifh.la?

mathomp4 · 2024-04-01T18:51:53Z

Here is what I see:


make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[3]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs/libopen-pal.so -ludev -ldl -lpthread -lrt -lm -lutil /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[3]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[3]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[2]: *** [Makefile:3436: all-recursive] Error 1
make[2]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make[1]: *** [Makefile:2801: all-recursive] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi'
make: *** [Makefile:1534: all-recursive] Error 1
mathomp4@gs6101-bucy .../FromGilles/openmpi-gitclone/build-nag-7.2.01

ggouaillardet · 2024-04-03T01:37:37Z

can you please

cd ompi/mpi/fortran/mpif-h/profile
rm *.la
make V=1

-Wl,-pthread does not get pulled with my nagfor symlink (!)

mathomp4 · 2024-04-03T15:31:32Z

@ggouaillardet If I run those three steps, all is well.

But if, from my build dir I do:

cd ompi/mpi/fortran/mpif-h
rm *.la
make V=1

I get the fail:

@$ make V=1 |& tee make.log
Making all in profile
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h/profile'
make[1]: Entering directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
/bin/sh ../../../../libtool  --tag=FC   --mode=link nagfor  -mismatch   -o libmpi_mpifh_sizeof.la  sizeof_f.lo  -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: rm -fr  .libs/libmpi_mpifh_sizeof.a .libs/libmpi_mpifh_sizeof.la
libtool: link: ar cr .libs/libmpi_mpifh_sizeof.a .libs/sizeof_f.o
libtool: link: ranlib .libs/libmpi_mpifh_sizeof.a
libtool: link: ( cd ".libs" && rm -f "libmpi_mpifh_sizeof.la" && ln -s "../libmpi_mpifh_sizeof.la" "libmpi_mpifh_sizeof.la" )
/bin/sh ../../../../libtool  --tag=CC   --mode=link gcc  -O3 -DNDEBUG  -finline-functions -mcx16 -version-info 80:0:40  -o libmpi_mpifh.la -rpath /ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib  ../../../../ompi/libmpi.la ../../../../ompi/mpiext/ftmpi/mpif-h/libmpiext_ftmpi_mpifh.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/libopen-pal.la profile/libmpi_mpifh_pmpi.la libmpi_mpifh_sizeof.la -lpthread -lrt -lm -lutil  /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_core.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/libevent_pthreads.la /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la
libtool: link: gcc -shared  -fPIC -DPIC  -Wl,--whole-archive ../../../../ompi/mpiext/ftmpi/mpif-h/.libs/libmpiext_ftmpi_mpifh.a profile/.libs/libmpi_mpifh_pmpi.a ./.libs/libmpi_mpifh_sizeof.a -Wl,--no-whole-archive  -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs -Wl,-rpath -Wl,/ford1/share/gmao_SIteam/MPI/openmpi/5.0.2-gilles/nag-7.2.01/lib ../../../../ompi/.libs/libmpi.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/opal/.libs/libopen-pal.so -ludev -ldl -lpthread -lrt -lm -lutil /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_core.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/libevent-2.1.12-stable/.libs/libevent_pthreads.so /ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/3rd-party/hwloc-2.7.1/hwloc/.libs/libhwloc.so  -O3 -mcx16   -pthread -Wl,-pthread -Wl,-soname -Wl,libmpi_mpifh.so.40 -o .libs/libmpi_mpifh.so.40.40.0
/usr/bin/ld: unrecognized option '-pthread'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:2938: libmpi_mpifh.la] Error 1
make[1]: Leaving directory '/ford1/share/gmao_SIteam/MPI/src/FromGilles/openmpi-gitclone/build-nag-7.2.01/ompi/mpi/fortran/mpif-h'
make: *** [Makefile:3436: all-recursive] Error 1
mathomp4@gs6101-bucy .../mpi/fortran/mpif-h

So it's like mpi/fortran/mpif-h/profile is good but not mpi/fortran/mpif-h

ggouaillardet · 2024-04-03T15:46:16Z

Thanks. I expected the three steps to go well, but I wanted to see how profile/libmpi_mpifh_psizeof.la was built, so can you please share the output?

mathomp4 · 2024-04-04T14:50:07Z

Ahhh. My apologies. Here you go:

profile_make_log.txt

ETA: This is just complex enough I didn't want to paste. 😄

ggouaillardet · 2024-04-05T00:05:31Z

Thanks, unfortunately I could not find anything relevant here ...

Instead of going back and forth, can you upload a tarball of your build directory? If too big, you can simply include all the config.log, Makefile and *.la files. I hope I will find where the -Wl,-pthread flags are pulled from.

mathomp4 · 2024-04-05T19:25:37Z

@ggouaillardet Here you go. This is my build dir but where I did --exclude=3rd-party to the tar command. Figured that was okay for a first try.

build-nag-7.2.01.tar.gz

ggouaillardet · 2024-04-07T04:06:55Z

Thanks again!

So I finally found the issue. -pthread is pulled from the internal libevent library (I failed to see this because I was using my distro's libevent that did not provide the *.la files.

I uploaded a new tarball (same location) https://github.com/ggouaillardet/ompi/releases/download/nag_pmix/openmpi-gitclone.tar.bz2

could you please give it a try?

mathomp4 · 2024-04-08T13:09:36Z

@ggouaillardet I grabbed that tarfile and it configures, it builds, it installs, it checks, and Hello World says "Hello World".¹

I think all is good!

Well, use mpi says "Hello World". use mpi_f08 is still broken with NAG. ↩

Fortran libraries do not need to be linked with PMIx, hwloc nor libevent. That can cause some issues with incompatible flags are pulled from the .la files. For example, when the NAG Fortran compiler is used, libmpi_mpifh_sizeof.la pulls the -pthread flag, that is translated into -Wl,-pthread. Then when libmpi_mpifh.la is linked with gcc but pulls -Wl,-pthread that causes the build to fail since -pthread is not a valid ld option. Not depending on these libraries is enough to avoid the above issue. Refs open-mpi#12413 Thanks to Matthew Thompson for reporting this issue and helping with the diagnostic. Signed-off-by: Gilles Gouaillardet <gilles@rist.or.jp>

ggouaillardet mentioned this issue Apr 9, 2024

fortran: do not link Fortran libs with unnecessary libraries #12458

Open

This was referenced Jul 24, 2024

MAPL3 + NAG + Bucy: MPI failure in some tests GEOS-ESM/MAPL#2921

Open

Build error with NAG 7.2: Option error: Unrecognised option -pthread pmodels/mpich#7080

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

mathomp4 commented Mar 18, 2024 •

edited

Loading

mathomp4 commented Mar 18, 2024

bosilca commented Mar 18, 2024

mathomp4 commented Mar 18, 2024 •

edited

Loading

ggouaillardet commented Mar 21, 2024

ggouaillardet commented Mar 21, 2024

mathomp4 commented Mar 21, 2024

ggouaillardet commented Mar 22, 2024

mathomp4 commented Mar 22, 2024

ggouaillardet commented Mar 23, 2024

ggouaillardet commented Mar 26, 2024

mathomp4 commented Mar 26, 2024 •

edited

Loading

ggouaillardet commented Apr 1, 2024

mathomp4 commented Apr 1, 2024

ggouaillardet commented Apr 1, 2024

mathomp4 commented Apr 1, 2024

ggouaillardet commented Apr 3, 2024

mathomp4 commented Apr 3, 2024

ggouaillardet commented Apr 3, 2024

mathomp4 commented Apr 4, 2024 •

edited

Loading

ggouaillardet commented Apr 5, 2024

mathomp4 commented Apr 5, 2024

ggouaillardet commented Apr 7, 2024

mathomp4 commented Apr 8, 2024

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

NAG 7.2 does not build 5.0.2, but does 4.1.6: -pthread passed to ld #12413

Comments

mathomp4 commented Mar 18, 2024 • edited Loading

Background information

What version of Open MPI are you using? (e.g., v4.1.6, v5.0.1, git branch name and hash, etc.)

Describe how Open MPI was installed (e.g., from a source/distribution tarball, from a git clone, from an operating system distribution package, etc.)

Please describe the system on which you are running

Details of the problem

mathomp4 commented Mar 18, 2024

bosilca commented Mar 18, 2024

mathomp4 commented Mar 18, 2024 • edited Loading

ggouaillardet commented Mar 21, 2024

ggouaillardet commented Mar 21, 2024

mathomp4 commented Mar 21, 2024

ggouaillardet commented Mar 22, 2024

mathomp4 commented Mar 22, 2024

ggouaillardet commented Mar 23, 2024

ggouaillardet commented Mar 26, 2024

mathomp4 commented Mar 26, 2024 • edited Loading

ggouaillardet commented Apr 1, 2024

mathomp4 commented Apr 1, 2024

ggouaillardet commented Apr 1, 2024

mathomp4 commented Apr 1, 2024

ggouaillardet commented Apr 3, 2024

mathomp4 commented Apr 3, 2024

ggouaillardet commented Apr 3, 2024

mathomp4 commented Apr 4, 2024 • edited Loading

ggouaillardet commented Apr 5, 2024

mathomp4 commented Apr 5, 2024

ggouaillardet commented Apr 7, 2024

mathomp4 commented Apr 8, 2024

Footnotes

mathomp4 commented Mar 18, 2024 •

edited

Loading

mathomp4 commented Mar 18, 2024 •

edited

Loading

mathomp4 commented Mar 26, 2024 •

edited

Loading

mathomp4 commented Apr 4, 2024 •

edited

Loading