Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[package] boost/1.76.0: cross-build to iOS fails #6490

Closed
SpaceIm opened this issue Jul 24, 2021 · 12 comments · Fixed by #7146
Closed

[package] boost/1.76.0: cross-build to iOS fails #6490

SpaceIm opened this issue Jul 24, 2021 · 12 comments · Fixed by #7146
Assignees
@SSE4
Labels
bug Something isn't working

Comments

@SpaceIm
Copy link
Contributor

SpaceIm commented Jul 24, 2021 •
edited
Loading

Package and Environment Details (include every applicable attribute)

  • Package Name/Version: boost/1.76.0
  • Operating System+version: Macos Catalina
  • Compiler+version: apple-clang 12.0
  • Conan version: conan 1.38.0
  • Python version: Python 3.9.6

Conan profile (output of conan profile show default or conan profile show <profile> if custom profile is in use)

profile-build

[settings]
os=Macos
arch=x86_64
compiler=apple-clang
compiler.version=12.0
compiler.libcxx=libc++
build_type=Release
[options]
[build_requires]
[env]

profile-host

[settings]
os=iOS
os.version=12.0
arch=armv8
compiler=apple-clang
compiler.version=12.0
compiler.libcxx=libc++
build_type=Release
[options]
[build_requires]
[env]

Steps to reproduce (Include if Applicable)

conan install boost/1.76.0@ -pr:b profile-build -pr:h profile-host -b missing

Logs (Include/Attach if Applicable)

Click to expand log 
boost/1.76.0: Applying build-requirement: b2/4.5.0
[HOOK - conan-center.py] pre_source(): [IMMUTABLE SOURCES (KB-H010)] OK
boost/1.76.0: Configuring sources in /Users/spaceim/.conan/data/boost/1.76.0/_/_/source
Downloading boost_1_76_0.tar.bz2 completed [107493.28k]

[HOOK - conan-center.py] post_source(): [LIBCXX MANAGEMENT (KB-H011)] OK
[HOOK - conan-center.py] post_source(): [CPPSTD MANAGEMENT (KB-H022)] OK
boost/1.76.0: Building your package in /Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452
boost/1.76.0: Generator txt created conanbuildinfo.txt
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] OK. 'fPIC' option found and apparently well managed
[HOOK - conan-center.py] pre_build(): [FPIC MANAGEMENT (KB-H007)] OK
boost/1.76.0: Calling build()
WARN: replace_in_file didn't find pattern '/* thread_local */' in '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/boost/stacktrace/detail/libbacktrace_impls.hpp' file.
WARN: replace_in_file didn't find pattern '/* static __thread */' in '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/boost/stacktrace/detail/libbacktrace_impls.hpp' file.
boost/1.76.0: WARN: Patching user-config.jam
boost/1.76.0: WARN:
using zlib : 1.2.11 : <include>"/Users/spaceim/.conan/data/zlib/1.2.11/_/_/package/108e68814533f28957889ee7075ad46a83968698/include" <search>"/Users/spaceim/.conan/data/zlib/1.2.11/_/_/package/108e68814533f28957889ee7075ad46a83968698/lib" <name>z ;
using bzip2 : 1.0.8 : <include>"/Users/spaceim/.conan/data/bzip2/1.0.8/_/_/package/402ac23a56db5265741eb8710725745ae6d028fa/include" <search>"/Users/spaceim/.conan/data/bzip2/1.0.8/_/_/package/402ac23a56db5265741eb8710725745ae6d028fa/lib" <name>bz2 ;
using "clang-darwin" :  :  /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.2.sdk -arch arm64 :
<archiver>"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ar" <ranlib>"/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib" <compileflags>"-I/Users/spaceim/.conan/data/libbacktrace/cci.20210118/_/_/package/108e68814533f28957889ee7075ad46a83968698/include" <linkflags>"-L/Users/spaceim/.conan/data/libbacktrace/cci.20210118/_/_/package/108e68814533f28957889ee7075ad46a83968698/lib"  ;
boost/1.76.0: Cross building, detecting compiler...
boost/1.76.0: Cross building flags: []
boost/1.76.0: WARN: b2 -q target-os=iphone architecture=arm address-model=64 binary-format=mach-o abi=aapcs --layout=system --user-config=/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-chrono --with-container --with-context --with-contract --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-locale --with-log --with-program_options --with-random --with-regex --with-serialization --with-stacktrace --with-system --with-test --with-thread --with-timer --with-type_erasure --with-wave toolset=clang-darwin pch=on linkflags="-stdlib=libc++" cxxflags="-fPIC -stdlib=libc++ -mios-version-min=12.0 -DBOOST_AC_USE_PTHREADS -DBOOST_SP_USE_PTHREADS -fembed-bitcode -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/Users/spaceim/.conan/data/boost/1.76.0/_/_/package/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452 -j12 --abbreviate-paths -d0 --debug-configuration --build-dir="/Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452"
notice: found boost-build.jam at /Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/boost-build.jam
notice: loading B2 from /Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/bootstrap.jam
warning: mismatched versions of B2 engine and core
warning: B2 engine (b2) is 4.5.0
warning: B2 core (at /Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src) is 4.4-git
notice: Searching '/etc' '/Users/spaceim' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/kernel' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/util' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/build' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/tools' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/contrib' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/.' for site-config configuration file 'site-config.jam'.
notice: Configuration file 'site-config.jam' not found in '/etc' '/Users/spaceim' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/kernel' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/util' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/build' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/tools' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/contrib' '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/src/.'.
notice: Loading explicitly specified user configuration file:
    /Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/user-config.jam
notice: Searching '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build' for user-config configuration file 'user-config.jam'.
notice: Loading user-config configuration file 'user-config.jam' from '/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build'.
notice: [zlib] Using pre-installed library
notice: [zlib] Condition
notice: [bzip2] Using pre-installed library
notice: [bzip2] Condition
notice: will use '/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++' '-isysroot' '/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.2.sdk' '-arch' 'arm64' for clang-darwin, condition <toolset>clang-12.0
notice: [zlib] zlib is already configured
notice: [bzip2] bzip is already configured
notice: iostreams: not using lzma compression
notice: iostreams: not using zstd compression
notice: [python-cfg] Configuring python...
notice: [python-cfg] Checking interpreter command "python"...
notice: [python-cfg] running command 'python -c "from sys import *; print('version=%d.%d\nplatform=%s\nprefix=%s\nexec_prefix=%s\nexecutable=%s' % (version_info[0],version_info[1],platform,prefix,exec_prefix,executable))" 2>&1'
notice: [python-cfg] ...requested configuration matched!
notice: [python-cfg] Details of this Python configuration:
notice: [python-cfg]   interpreter command: "python"
notice: [python-cfg]   include path: "/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7"
notice: [python-cfg]   library path: "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config" "/System/Library/Frameworks/Python.framework/Versions/2.7/lib"
notice: [python-cfg] Checking for NumPy...
notice: [python-cfg] running command 'python -c "import sys; sys.stderr = sys.stdout; import numpy; print(numpy.get_include())"'
notice: [python-cfg] NumPy enabled
notice: [python-cfg] framework directory is "/System/Library/Frameworks/Python.framework"
Performing configuration checks

    - default address-model    : 64-bit [1]
    - default architecture     : arm [1]
    - C++11 mutex              : no [2]
    - lockfree boost::atomic_flag : no [2]
    - has stat::st_mtim        : no [2]
    - has stat::st_mtimensec   : no [2]
    - has stat::st_mtimespec   : yes [2]
    - has stat::st_birthtim    : no [2]
    - has stat::st_birthtimensec : no [2]
    - has stat::st_birthtimespec : yes [2]
    - has statx                : no [2]
    - has statx syscall        : no [2]
    - zlib                     : no [3]
    - bzip2                    : no [3]
    - iconv (libc)             : no [2]
- Boost.Locale failed to find iconv (libc) library to be built.
boost/1.76.0:
boost/1.76.0: ERROR: Package '7ab220ce003d7c6ffe8c029987b8bf98cb7ba452' build failed
boost/1.76.0: WARN: Build folder /Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452
ERROR: boost/1.76.0: Error in build() method, line 801
	self.run(full_command, run_environment=True)
	ConanException: Error 1 while executing DYLD_LIBRARY_PATH="" DYLD_FRAMEWORK_PATH="" b2 -q target-os=iphone architecture=arm address-model=64 binary-format=mach-o abi=aapcs --layout=system --user-config=/Users/spaceim/.conan/data/boost/1.76.0/_/_/source/source_subfolder/tools/build/user-config.jam -sNO_ZLIB=0 -sNO_BZIP2=0 -sNO_LZMA=1 -sNO_ZSTD=1 boost.locale.icu=off --disable-icu boost.locale.iconv=on boost.locale.iconv.lib=libc threading=multi visibility=hidden link=static variant=release --with-atomic --with-chrono --with-container --with-context --with-contract --with-coroutine --with-date_time --with-exception --with-filesystem --with-iostreams --with-locale --with-log --with-program_options --with-random --with-regex --with-serialization --with-stacktrace --with-system --with-test --with-thread --with-timer --with-type_erasure --with-wave toolset=clang-darwin pch=on linkflags="-stdlib=libc++" cxxflags="-fPIC -stdlib=libc++ -mios-version-min=12.0 -DBOOST_AC_USE_PTHREADS -DBOOST_SP_USE_PTHREADS -fembed-bitcode -DBOOST_STACKTRACE_ADDR2LINE_LOCATION=/usr/bin/addr2line" install --prefix=/Users/spaceim/.conan/data/boost/1.76.0/_/_/package/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452 -j12 --abbreviate-paths -d0 --debug-configuration --build-dir="/Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452"

If I use libiconv backend instead of libc (by replacing this line

conan-center-index/recipes/boost/all/conanfile.py

Line 275 in 27f4591

elif self.settings.os == "Macos":
by tools.is_apple_os(self.settings.os)), it also fails.
Dependencies are not found (see the summary, bzip2 and zlib also are not found).
@SpaceIm SpaceIm added the bug Something isn't working label Jul 24, 2021
@madebr
Copy link
Contributor

madebr commented Jul 25, 2021

Try increasing the debug level (it's an option), perhaps a verbose log contains more information.

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jul 27, 2021

here is config.log with debug level increased

config.log

@SSE4
Copy link
Contributor

SSE4 commented Jul 27, 2021

that seems to be an actual error:

clang-darwin.link /Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452/boost/bin.v2/standalone/ac/clng-drwn-12.0/rls/trgt-os-iphn/thrd-mlt/vsblt-hdn/bz2

    "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.2.sdk" "-arch" "arm64" -stdlib=libc++ -L"/Users/spaceim/.conan/data/bzip2/1.0.8/_/_/package/402ac23a56db5265741eb8710725745ae6d028fa/lib" -o "/Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452/boost/bin.v2/standalone/ac/clng-drwn-12.0/rls/trgt-os-iphn/thrd-mlt/vsblt-hdn/bz2" "/Users/spaceim/.conan/data/boost/1.76.0/_/_/build/7ab220ce003d7c6ffe8c029987b8bf98cb7ba452/boost/bin.v2/standalone/ac/clng-drwn-12.0/rls/trgt-os-iphn/thrd-mlt/vsblt-hdn/main-bz2.o"  -Wl,--start-group -Wl,-Bstatic  -Wl,-Bdynamic -lrt -lbz2 -Wl,--end-group -fPIC -pthread -fvisibility=hidden -fvisibility-inlines-hidden -L/Users/spaceim/.conan/data/libbacktrace/cci.20210118/_/_/package/108e68814533f28957889ee7075ad46a83968698/lib

ld: unknown option: --start-group
clang: error: linker command failed with exit code 1 (use -v to see invocation)

I think I've seen similar issue with --start-group in the past

@SSE4
Copy link
Contributor

SSE4 commented Jul 27, 2021

that's one suspicious part: <link>shared
AFAIK iOS doesn't support shared libraries

@SSE4
Copy link
Contributor

SSE4 commented Jul 27, 2021

see also:
boostorg/build#365
boostorg/build#678
boostorg/build#438
#3992
/cc @grafikrobot

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jul 27, 2021

Yes, shared is not supported on iOS

@SSE4
Copy link
Contributor

SSE4 commented Jul 27, 2021

Yes, shared is not supported on iOS

so, why are you passing <link>shared? I don't see shared=True in your profile

@SpaceIm
Copy link
Contributor Author

SpaceIm commented Jul 27, 2021 •
edited
Loading

Why would I have passed this kind of flags? 😄 I'm not passing anything myself. My conan install command is simple, without any trick: #6490 (comment)

@SSE4
Copy link
Contributor

SSE4 commented Jul 27, 2021

it seems to be going from there:
https://github.com/boostorg/locale/blob/develop/build/Jamfile.v2#L37

I don't know why doesn't it fail for regular macOS, which should use the same rule to link (thus, using --start-group flag)?

@SSE4 SSE4 self-assigned this Jul 27, 2021
@SpaceIm SpaceIm mentioned this issue Aug 7, 2021
Closed
@leha-bot
Copy link
Contributor

leha-bot commented Aug 10, 2021 •
edited
Loading

Hello, guys, I got a similar problem recently while testing my own recipe in termux at Android device with the similar error.

The very interesting in that story that the upstream Boost have found the Termux's libiconv and ICU! (at least at boosttrapping stage via B2 and (suddenly!!) CMake)

boosttrap.sh log 
~/boost $ ./bootstrap.sh --prefix=~/boost-b2-build            Building B2 engine..

###
###
### Using 'gcc' toolset.
###
###

clang version 12.0.1
Target: aarch64-unknown-linux-android24
Thread model: posix
InstalledDir: /data/data/com.termux/files/usr/bin

###
###

> g++ -x c++ -std=c++11 -O2 -s -DNDEBUG builtins.cpp class.cpp command.cpp compile.cpp constants.cpp cwd.cpp debug.cpp debugger.cpp execcmd.cpp execnt.cpp execunix.cpp filesys.cpp filent.cpp fileunix.cpp frames.cpp function.cpp glob.cpp hash.cpp hcache.cpp hdrmacro.cpp headers.cpp jam_strings.cpp jam.cpp jamgram.cpp lists.cpp make.cpp make1.cpp md5.cpp mem.cpp modules.cpp native.cpp object.cpp option.cpp output.cpp parse.cpp pathnt.cpp pathsys.cpp pathunix.cpp regexp.cpp rules.cpp scan.cpp search.cpp startup.cpp subst.cpp sysinfo.cpp timestamp.cpp variable.cpp w32_getreg.cpp modules/order.cpp modules/path.cpp modules/property-set.cpp modules/regex.cpp modules/sequence.cpp modules/set.cpp -o b2
> cp b2 bjam
tools/build/src/engine/b2
Detecting Python version... 3.9
Detecting Python root... /data/data/com.termux/files/usr
Unicode/ICU support for Boost.Regex?... /data/data/com.termux/files/usr
Generating B2 configuration in project-config.jam for gcc...

Bootstrapping is done. To build, run:

    ./b2

To generate header files, run:

    ./b2 headers

The configuration generated uses gcc to build by default. If that is
unintended either use the --with-toolset option or adjust configuration, by
editing 'project-config.jam'.

Further information:

   - Command line help:
     ./b2 --help

   - Getting started guide:
     http://www.boost.org/more/getting_started/unix-variants.html

   - B2 documentation:
     http://www.boost.org/build/

~/boost $

@SSE4
Copy link
Contributor

SSE4 commented Sep 1, 2021

debugging further, comparing iOS and macOS builds...
so, the problem goes from there:
https://github.com/boostorg/build/blob/boost-1.77.0/src/tools/gcc.jam#L887
as you can see, START_GROUP is set to -Wl,--start-group for generic os (and, perhaps, an empty otherwise)
and generic os is set here:
https://github.com/boostorg/build/blob/boost-1.77.0/src/tools/gcc.jam#L880
so, darwin is generic (the one we use for macOS), while iphone is not (the one we use for iOS)
let's see if we can somehow override START_GROUP from within user-config.jam...

@SSE4
Copy link
Contributor

SSE4 commented Sep 2, 2021 •
edited
Loading

the following simple patch:

diff --git a/recipes/boost/all/conanfile.py b/recipes/boost/all/conanfile.py
index c791c912f..3f5c2dabe 100644
--- a/recipes/boost/all/conanfile.py
+++ b/recipes/boost/all/conanfile.py
@@ -778,6 +778,14 @@ class BoostConan(ConanFile):
                                   "thread_local", "/* thread_local */")
             tools.replace_in_file(os.path.join(self.source_folder, self._source_subfolder, "boost", "stacktrace", "detail", "libbacktrace_impls.hpp"),
                                   "static __thread", "/* static __thread */")
+        tools.replace_in_file(os.path.join(self.source_folder, self._source_subfolder, "tools", "build", "src", "tools", "gcc.jam"),
+                              "local generic-os = [ set.difference $(all-os) : aix darwin vxworks solaris osf hpux ] ;",
+                              "local generic-os = [ set.difference $(all-os) : aix darwin vxworks solaris osf hpux iphone ] ;",
+                              strict=False)
+        tools.replace_in_file(os.path.join(self.source_folder, self._source_subfolder, "tools", "build", "src", "tools", "gcc.jam"),
+                              "local no-threading = android beos haiku sgi darwin vxworks ;",
+                              "local no-threading = android beos haiku sgi darwin vxworks iphone ;",
+                              strict=False)

         if self.options.header_only:
             self.output.warn("Header only package, skipping build")
(END)

goes a bit further:

1292     "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++" "-isysroot" "/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS14.5.sdk" "-arch" "arm6     4" -stdlib=libc++  -o "/Users/sse4/.conan/data/boost/1.76.0/_/_/build/0b7699390061954d7f4fc6b2006a8dcebb46d40b/boost/bin.v2/libs/locale/build/clng-drwn-12.0/rls/lnk-sttc/trgt-os-iphn/thrd-mlt/vsblt-hdn/has_iconv" "/Users/sse4/.cona     n/data/boost/1.76.0/_/_/build/0b7699390061954d7f4fc6b2006a8dcebb46d40b/boost/bin.v2/libs/locale/build/clng-drwn-12.0/rls/lnk-sttc/trgt-os-iphn/thrd-mlt/vsblt-hdn/has_iconv_libc_obj.o"        -fvisibility=hidden -fvisibility-inlines     -hidden
1293
1294 Undefined symbols for architecture arm64:
1295   "_iconv_open", referenced from:
1296       _main in has_iconv_libc_obj.o
1297 ld: symbol(s) not found for architecture arm64

next thing is need to understand if iOS libc provides iconv_open, or do we need to specify external iconv backend in this case?

This was referenced Sep 2, 2021
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@SSE4 SSE4

Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

- [boost] fix libiconv detection (needed by boost locale) for iOS, Android SSE4/conan-center-index
4 participants
@SSE4 @madebr @leha-bot @SpaceIm