Update conan packages on our NREL conan Artifactory remote (for MSVC 2022 and mac M1 in particular)

[jmarrec]

Issue overview

Updated packages so far:

• pcre/8.41@#1afabf78c46a11bdd1755152e44f6e59

• bzip2/1.0.8@#9c8b632000a52af5af3f045d3a6db12f

• libffi/3.3@#cf635daae2a4fba32eeb982715df9305

• gdbm/1.18.1@#ecdb7cd9673803e86868bf3c493c6fa7

• b2/4.2.0@#a45bb9ebe2e994daaf83842b5b682eea

• bison/3.7.1@#ad29e804e82c8b6d58765096676b5a5e

• flex/2.6.4@#e4696e6333f2d486e7073de6a9559712 (due to mistmatch of m4)

• gmp/6.2.0@#30cb7f744db8ac600167b05f747178c7 (due to mistmatch of m4)

  'flex/2.6.4' requires 'm4/1.4.18' while 'bison/3.7.1' requires 'm4/1.4.19'.
  

• termcap/1.3.1@#a66ace38d82bcfa0c0b39e9d4d0c71d6 (still chasing the mismatch of m4)

• bison/3.5.3@#01aa231aae5441239d4fc697384d1811 : the mistmatch of m4 was coming from there actually...

• libyaml/0.2.5@#edaee18262dfefa593ab650babefc80e

• readline/8.0@#1d21941e543bef7a06551f5a56c8a87e

• autoconf/2.69@# 5f19ba50aed010d74626c5281c65f135 ('autoconf/2.69' requires 'm4/1.4.18' while 'bison/3.7.1' requires 'm4/1.4.19')

• swig/4.0.2@#187cec21277aafbeae11dcc3b936ce74: since this commit, swig has the correct swiglib so no need to do shenanigans with FindValue and passing SWIG_LIB env variable and co!

  • Found out that's not true on mac: opened [package] swig/4.0.2: SWIG_DIR (swiglib) is wrong on mac and points to the build machine conan-io/conan-center-index#8674 and swig/4.0.2: Replace get_exe_path by something that works on both Ubuntu and macOS conan-io/conan-center-index#8675 to fix it

Need to replace the pcre recipes we have on our NREL conan Artifactory remote

The old ftp addresses where the pcre tar.gz were hosted are now dead. This is fixed in conan-center, but NREL's artifactory remote uses the old recipes that still point to it.

I got this issue on a clean mac M1 machine (any machine that already have the pcre 8.41 installed from conan will be ok). Edit: I just hit this today on Ubuntu 20.04 with a clean cache

conan-io/conan-center-index#8103

Steps to Reproduce

Remove prce from ~/.conan/data. Try to build openstudio (or conan-openstudio-ruby), which places NREL's remote first in line -> it fails

ERROR: Error downloading file https://ftp.pcre.org/pub/pcre/pcre-8.41.tar.gz: 'HTTPSConnectionPool(host='ftp.pcre.org', port=443): Max retries exceeded with url: /pub/pcre/pcre-8.41.tar.gz (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x1082a37c0>, 'Connection to ftp.pcre.org timed out. (connect timeout=60.0)'))'

Possible Solution

Remove pcre package, reupload the current from conan-center-index.

Details

Environment

Some additional details about your environment for this issue (if relevant):

• Platform (Operating system, version): M1 mac

[jmarrec]

Same applies to bzip2, for m1 arm64 support. EDIT: bzip2 is fine, it's just that is depends on pcre I think.

[ 98%] Building C object source_subfolder/CMakeFiles/pcregrep.dir/pcregrep.c.o
[100%] Linking C executable ../bin/pcregrep
Undefined symbols for architecture arm64:
  "_BZ2_bzclose", referenced from:
      _grep_or_recurse in pcregrep.c.o
  "_BZ2_bzopen", referenced from:
      _grep_or_recurse in pcregrep.c.o
  "_BZ2_bzread", referenced from:
      _pcregrep in pcregrep.c.o
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/pcregrep] Error 1
make[1]: *** [source_subfolder/CMakeFiles/pcregrep.dir/all] Error 2
make: *** [all] Error 2
pcre/8.41: 
pcre/8.41: ERROR: Package '292f9959fdc8eda5c3b7b35cab5f9386c240c028' build failed
pcre/8.41: WARN: Build folder /Users/julien/.conan/data/pcre/8.41/_/_/build/292f9959fdc8eda5c3b7b35cab5f9386c240c028
ERROR: pcre/8.41: Error in build() method, line 138
	cmake.build()
	ConanException: Error 2 while executing cmake --build '/Users/julien/.conan/data/pcre/8.41/_/_/build/292f9959fdc8eda5c3b7b35cab5f9386c240c028/build_subfolder' '--' '-j10'

[jmarrec]

Similar issue with libffi for M1 macs.

[jmarrec]

Packages on our remote are also missing all MSVC 2022 binaries, forcing build on the host machine

The b2 package doesn't build (it has to build since we do not have 2022 packages on the remote) correctly with MSVC 2022 either. Apparently that one needs the MSVC 2017 (vc142) to build

---

Workaround for now

Basically force download it from conancenter instead of NREL's.

mkdir tmp && cd tmp
/bin/rm -Rf ~/.conan/data/b2/
echo "[requires]" > conanfile.txt
echo "b2/4.2.0" >> conanfile.txt
conan remote add --force --insert 0 conancenter https://center.conan.io
conan install .
conan remote add --force --insert 1 conancenter https://center.conan.io

[jmarrec]

boost/1.73.0#4129a76c9b83c300fc103e36d1908792 won't work with MSVC 2022 either.

Trying boost/1.73.0#9c4e07063572e776be60507c405b6754 from conancenter.io now... This doesn't have a MSVC 2022 package prebuilt either. I think most (if not all) recipes on conancenter don't have a MSVC 2022 package yet

Maybe now would be a good time to bump to boost 1.78?

---

Testing all versions from conancenter until one builds...

conan remote add --force --insert 0 conancenter https://center.conan.io
rm ~/.conan/data/boost/
conan install boost/1.74.0@ -b missing  # Fails as well
conan install boost/1.77.0@ -b missing  # Fails as well
conan install boost/1.78.0@ -b missing  # Works

Seems like one needs boost 1.78 to have MSVC 2022 support. This confirms what I had found searching online, that's officially the first version of boost that natively supports MSVC 2022 (I have found someone who manually patched 1.77 to make it work, but that's not in conan)

[jmarrec]

Ubuntu 20.04, clean cache. Hit the pcre issue with the NREL package. I delete pcre, put the conancenter remote first in line. It tries to build it, then fails with bzip2, so there is a problem with bzip2 indeed

[100%] Linking C executable ../bin/pcregrep
/usr/bin/ld: CMakeFiles/pcregrep.dir/pcregrep.c.o: in function `pcregrep':
pcregrep.c:(.text+0xfb6): undefined reference to `BZ2_bzread'
/usr/bin/ld: pcregrep.c:(.text+0x20d4): undefined reference to `BZ2_bzread'
/usr/bin/ld: CMakeFiles/pcregrep.dir/pcregrep.c.o: in function `grep_or_recurse.part.0.isra.0':
pcregrep.c:(.text+0x2c2f): undefined reference to `BZ2_bzopen'
/usr/bin/ld: pcregrep.c:(.text+0x2cb1): undefined reference to `BZ2_bzclose'
/usr/bin/ld: pcregrep.c:(.text+0x2e19): undefined reference to `BZ2_bzerror'
/usr/bin/ld: pcregrep.c:(.text+0x2ed8): undefined reference to `BZ2_bzclose'
collect2: error: ld returned 1 exit status
make[2]: *** [source_subfolder/CMakeFiles/pcregrep.dir/build.make:100: bin/pcregrep] Error 1
make[1]: *** [CMakeFiles/Makefile2:211: source_subfolder/CMakeFiles/pcregrep.dir/all] Error 2
make: *** [Makefile:136: all] Error 2
pcre/8.41: 
pcre/8.41: ERROR: Package '9e019e5a472de44f8afd9be5d77633d599850649' build failed
pcre/8.41: WARN: Build folder /home/julien/.conan/data/pcre/8.41/_/_/build/9e019e5a472de44f8afd9be5d77633d599850649

This fixed it:

/bin/rm -Rf ~/.conan/data/bzip2/
conan install pcre/8.41@ -b missing

bzip2/1.0.8#c33162705fba3ed3998038dd7296d96d is what I got out it.

Then I tried to compile conan-openstudio-ruby, hit an issue with gdbm1/18.1 from nrel. conan install -r conancenter gdbm/1.18.1@ -b missing fixed it, I got ecdb7cd9673803e86868bf3c493c6fa7

[tijcolem]

Thanks for the report @jmarrec

If we can summarize any broken dependencies for the currently support platforms, I can upload these to our NREL conan site if you haven't done so already. If these are only for M1 and MSVC 2022, that's fine too but maybe we can change the title to indicate it's only for these two and we will work to update to support them. It'll take a little time to get MSVC 2022 up and running.

Platform	Conan packages that need updating
MSVC 2019
OSX 10.15
Ubuntu 18.04
Ubuntu 20.04
OSX M1
MSVC 2022