Boost 1.76 and 1.77 (beta) don't create static libraries on macOS with build-type=complete

[lazar-ivanov]

Boost 1.76 and 1.77 Boost.Locale doesn't create the static libraries (s, sd flavors) with build-type=complete on macOS (Mojave, Bug Sur, both arm & intel). I'm using boost.locale.icu=off boost.locale.iconv=on parameters and you can see the complete b2 command below.

On Boost 1.75 it works and creates the following libraries:

libboost_locale-mt-a64.a
libboost_locale-mt-a64.dylib
libboost_locale-mt-d-a64.a
libboost_locale-mt-d-a64.dylib
libboost_locale-mt-s-a64.a
libboost_locale-mt-sd-a64.a

On Boost 1.76 and 1.77 does not create the static versions (sd & s flavors), but just the dynamic versions:

libboost_locale-mt-a64.a
libboost_locale-mt-a64.dylib
libboost_locale-mt-d-a64.a
libboost_locale-mt-d-a64.dylib

The complete b2 command used is the following:

./b2 --prefix=./bld --stagedir=./bld/stage --build-dir=./bld/int -j16 --build-type=complete --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxflags="-stdlib=libc++ -std=c++11 -fPIC" linkflags="-stdlib=libc++ -std=c++11 -fPIC" --without-python --without-iostreams boost.locale.icu=off boost.locale.iconv=on install > ./build.log

[lazar-ivanov]

Forgot to note that the build does find iconv (below):

Performing configuration checks

- default address-model    : 64-bit
- default architecture     : arm
- C++11 mutex              : yes
- lockfree boost::atomic_flag : yes
- has stat::st_mtim        : no
- has stat::st_mtimensec   : no
- has stat::st_mtimespec   : yes
- has stat::st_birthtim    : no
- has stat::st_birthtimensec : no
- has stat::st_birthtimespec : yes
- has statx                : no
- has statx syscall        : no
- cxx11_auto_declarations  : yes
- cxx11_constexpr          : yes
- cxx11_defaulted_functions : yes
- cxx11_final              : yes
- cxx11_hdr_mutex          : yes
- cxx11_hdr_tuple          : yes
- cxx11_lambdas            : yes
- cxx11_noexcept           : yes
- cxx11_nullptr            : yes
- cxx11_rvalue_references  : yes
- cxx11_template_aliases   : yes
- cxx11_thread_local       : yes
- cxx11_variadic_templates : yes
- has_icu builds           : no

warning: Graph library does not contain MPI-based parallel components.
note: to enable them, add "using mpi ;" to your user-config.jam.
note: to suppress this message, pass "--without-graph_parallel" to bjam.
- cxx11_alignas : yes
- cxx11_decltype : yes
- iconv (libc) : no
- iconv (separate) : yes
- native atomic int32 supported : yes
- native syslog supported : yes
- pthread supports robust mutexes : no
- gcc visibility : yes
- long double support : yes
warning: skipping optional Message Passing Interface (MPI) library.
note: to enable MPI support, add "using mpi ;" to user-config.jam.
note: to suppress this message, pass "--without-mpi" to bjam.
note: otherwise, you can safely ignore this message.
- cxx11_static_assert : yes
- std::fstream is moveable and swappable : yes
- Has Large File Support : yes
- libbacktrace builds : no
- addr2line builds : yes
- WinDbg builds : no
- WinDbgCached builds : no
- BOOST_COMP_GNUC >= 4.3.0 : no

[Flamefire]

Cannot reproduce, must be something on your end.
BTW: Instead of your cxxflags I suggest to use cxxstd=11 stdlib=libc++, maybe that is enough to solve your issue or point to the real problem (e.g. no static libc++?)

[lazar-ivanov]

Can you elaborate on what do you mean that you can't reproduce? Have you tried the b2 command above? Are you suggesting it is because of the cxxflags parameters? This happens for me on any of the Macs I have with 1.76 and not happening with 1.75.

If you simply bootstrap.sh followed by the b2 command above should be easily reproducible.

[lazar-ivanov]

I also tried this on brand new Mac and it happens here too. The only thing I have installed on the Mac was Xcode + the command line tools which is of course needed to bring the toolchain.

[Flamefire]

Can you elaborate on what do you mean that you can't reproduce?

Doesn't happen for me on Linux with the command provided

Are you suggesting it is because of the cxxflags parameters?

Only guessing here but yes and especially it may "point to the real problem (e.g. no static libc++?)"

I have with 1.76 and not happening with 1.75.

Is Boost.Locale the only library affected? Because it is literally the same for those 2 versions.

Can you try again with 1.76 and attach the build log? It should rather look like - cxx11_auto_declarations : no [2] (i.e. have those numbers for the different configurations)

[lazar-ivanov]

It is entirely possible that it doesn't happen on Linux. It is a different platform and different environment. I have not tried building it for Linux (yet), but I'm able to reproduce it on macOS 100% consistently and also with different macOS versions and clang toolchains. E.g. I can reproduce it on somewhat older intel Mac with macOS Mojave and Clang 10.0.0 and very recent arm Mac (m1) with macOS Big Sur and clang 12.0.5.

I can try running with the parameters you specify and will let you know, but the command line I'm using should be legit and it should work fine.

Is Boost.Locale the only library affected - no, but only one other library is affected - one of the libraries created by Boost.Stacktrace also has the same problem, but all other libraries are ok.

I will attach the build.log file in my next comment in a couple of days.

[lazar-ivanov]

I did check on Linux and this problem does not happen there, but as I mentioned above this is not surprising as Linux is quite a different env than macOS (including e.g. iconv is part of libc there unlike on the Mac). I also did try with the cxxstd=11 stdlib=libc++ parameters and this did not help. The - cxx11_auto_declarations in the log file is "yes". Attached are the two log files - one with the original b2 command line and the second with the cxxstd=11 stdlib=libc++ parameters (replacing the respective cxxflags and linkflags parameters).
build-log-with-cxxstd-stdlib-parameters.log
build-log-with-original-parameters.log

[Flamefire]

    - iconv (libc)             : no [10]
    - iconv (separate)         : no [10]
- Boost.Locale needs either iconv or ICU library to be built.
    - iconv (libc)             : no [11]
    - iconv (separate)         : no [11]
- Boost.Locale needs either iconv or ICU library to be built.

That is a good hint. Can you create another log with: ./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=static -d+2 --reconfigure -a --debug-building?

[lazar-ivanov]

Attached is the log with the command line above
build-log-with-debug-parameters.log

[Flamefire]

Sorry, digging in the dark here and I'm not to familiar with b2 myself so need to ask you for some more info.
Can you rerun the above command with runtime-link=shared instead of the static and for both attach also the config.log from bld/int/boost/bin.v2/?

[lazar-ivanov]

No worries and apologies for the delayed replies. Thank you for investigating it. See attached files (both build and config logs for the static and shared).
build-log-with-debug-parameters-shared.log
config-log-with-debug-parameters-shared.log
config-log-with-debug-parameters-static.log
build-log-with-debug-parameters-static.log

[Flamefire]

Ok checked that and got (shortened) the following as the difference:

"clang++" -o "has_iconv" "has_iconv_libc_obj.o" -std=c++11 -g -fvisibility=hidden -fvisibility-inlines-hidden -static -stdlib=libc++

"clang++" -o "has_iconv" "has_iconv_libc_obj.o" -std=c++11 -g -fvisibility=hidden -fvisibility-inlines-hidden -stdlib=libc++

As you can see the invocation from B2 is fully correct as far as I can tell.

The error from the static build is: ld: library not found for -lcrt0.o which according to https://stackoverflow.com/questions/3801011/ld-library-not-found-for-lcrt0-o-on-osx-10-6-with-gcc-clang-static-flag hints that fully static builds on OSX are plain unsupported in most scenarios.

--> We can't do much, maybe you can install/change something on your system so this works.

As you said it has seemingly worked in 1.75, would you mind creating the above logs for that Boost version and on the same system too? I suspect either that it hasn't actually worked or something on your system has changed.

[lazar-ivanov]

I'm not sure what do you mean by "fully static builds", usually what it means is to link statically to the C++ standard library (libc++ or libstdc++), but not to link statically to libc or other OS level libraries. I believe this is also what this boost option means (runtume-link -- a.k.a. the c++ std lib runtime). I'm not sure why it fails to find this file and why the -static parameter works this way, but it clearly something with Boost.Locale because all other boost libraries (except one) are building and producing static libraries just fine. Also it builds and produces static libraries for 1.75, so it is broken only for Boost.Locale and only for 1.76 and onwards. Let me know if you want me to build some other library to diff and try to see how they are linking. Attached are the build and config files for 1.75 that you requested (for both static and shared command line).
1.75-build-log-with-debug-parameters-shared.log
1.75-config-log-with-debug-parameters-shared.log
1.75-build-log-with-debug-parameters-static.log
1.75-config-log-with-debug-parameters-static.log

[lazar-ivanov]

BTW, I will be traveling in the next week or so (in case I'm not able to respond), just fyi-

[Flamefire]

Just to be sure: Have you used the exact same b2 cmdline for the 1.75 build?

The configure checks:
in 1.75:
"clang++" -o ".../has_iconv" ".../has_iconv_libc_obj.o" -fPIC -std=c++11 -g -fvisibility=hidden -fvisibility-inlines-hidden

In 1.76:
"clang++" -o ".../has_iconv" ".../has_iconv_libc_obj.o" -std=c++11 -g -fvisibility=hidden -fvisibility-inlines-hidden -static -stdlib=libc++

I.e. missing the -fPIC and seemingly the -static.
I'll check back with the b2 author but it is possible, that there is a bug newly introduced or fixed, so not sure what is "correct"

[lazar-ivanov]

Yes, I have used the same exact command lines for the 1.75 logs I sent above (see command lines below). I'm not sure why 1.76 is adding -static instead of -fPIC.

./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=static -d+2 --reconfigure -a --debug-building > ./build.log
./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=shared -d+2 --reconfigure -a --debug-building > ./build.log

[Flamefire]

Ok, the -static flag added to the configure check is due to a bugfix in recent B2 and should have been there before.
Question is why it did work before because I'd expect that linking the library would fail

Can you try ./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=static variant=shared -d+2 --reconfigure -a --debug-building > ./build.log on 1.75 please?
It should contain the commands used to compile and link the lib. I suspect it is missing the -static flag too. Or something else...

[lazar-ivanov]

On 1.75 this command line has failed with an error saying that variant=shared is invalid (see below & attached file, config file is empty):

/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build/feature.jam:491: in feature.validate-value-string from module feature
error: "shared" is not a known value of feature
error: legal values: "debug" "release" "profile" "debug-python"
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build/property.jam:333: in validate1 from module property
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build/property.jam:359: in property.validate from module property
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build/build-request.jam:271: in convert-command-line-element from module build-request
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build/build-request.jam:222: in build-request.convert-command-line-elements from module build-request
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/build-system.jam:774: in load from module build-system
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/kernel/modules.jam:295: in import from module modules
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/tools/build/src/kernel/bootstrap.jam:139: in boost-build from module
/Users/lazar/swblocks/dist-devenv5-darwin-20-arm/scratch/boost/boost_1_75_0/boost-build.jam:17: in module scope from module

1.75-build-log-with-debug-parameters-static-2.log

[Flamefire]

Oh, my bad: It should be link=shared: ./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=static link=shared -d+2 --reconfigure -a --debug-building > ./build.log

[lazar-ivanov]

Is someone interested to look into this issue? This is still happening with boost 1.84 on macOS Ventura and it is very frustrating because effectively you can't use the boost locale library if you are linking statically.

[lazar-ivanov]

The folder under runtime-link-static is basically empty - see attached screenshot

[Flamefire]

I'd need the build logs each for 1.75 and newer created by using ./b2 --build-dir=./bld/int --with-locale --layout=tagged runtime-debugging=off debug-symbols=on address-model=64 cxxstd=11 stdlib=libc++ runtime-link=static link=shared -d+2 --reconfigure -a --debug-building > ./build.log to investigate why it fails on your system.