Bumps deps by jmarrec · Pull Request #39 · NatLabRockies/conan-openstudio-ruby

jmarrec · 2022-04-01T10:06:32Z

Companion PR: NatLabRockies/OpenStudio#4596

Latest run: https://github.com/jmarrec/conan-openstudio-ruby/runs/6564537150?check_suite_focus=true

Original Post:

It's running on my fork, https://github.com/jmarrec/conan-openstudio-ruby/runs/5785999801?check_suite_focus=true

I have an issue...:

openssl 1.1.1n won't build on GCC 7 and 8 on CI (it works fine locally on my ubuntu 20.04 machine...)

 /home/conan/.conan/data/openssl/1.1.1n/_/_/package/d50a0d523d98c15bb147b18fa7d203887c38be8b/lib/libcrypto.a(threads_pthread.o): In function `fork_once_func':
threads_pthread.c:(.text+0x16): undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status

openssl 1.1.0l won't build in debug on msvc-2022 (and there is no binary package for it):

2022-04-01T10:39:08.7868654Z 	set ASM=nasm
2022-04-01T10:39:08.7869883Z 	"C:\.conan\18ba5c\1\bin\perl.exe" "crypto\aes\asm\aes-x86_64.pl" auto crypto\aes\aes-x86_64.asm
2022-04-01T10:39:09.0013273Z 	nasm -f win64 -DNEAR -Ox -g -ocrypto\aes\aes-x86_64.obj "crypto\aes\aes-x86_64.asm"
2022-04-01T10:39:09.8242544Z NMAKE : fatal error U1077: 'C:\Users\runneradmin\.conan\data\nasm\2.14\_\_\package\01edd76db8e16db9b38c3cca44ec466a9444c388\bin\nasm.EXE' : return code '0xc0000005'
2022-04-01T10:39:09.8243466Z Stop.
2022-04-01T10:39:09.8274678Z NMAKE : fatal error U1077: '"C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.31.31103\bin\HostX64\x64\nmake.exe"' : return code '0x2'
2022-04-01T10:39:09.8275272Z Stop.

…as 1.x and 3.x, 3.x does not work) ruby/openssl#369

conancenter still isn't picked up What the hell? Debug why it's not creating SWIGRubyRuntime.hxx Missing dependency chain in windows Re-enable other platforms

``` linking miniruby.exe parse.obj : error LNK2019: unresolved external symbol YYUSE referenced in function rb_yytnamerr [C:\Users\julien\.conan \data\openstudio_ruby\2.7.2\nrel\testing\build\45b3b9a1aa73fdb3ff32afdc2b2ddc0affca7928\Ruby.vcxproj] miniruby.exe : fatal error LNK1120: 1 unresolved externals [C:\Users\julien\.conan\data\openstudio_ruby\2.7.2\nrel\test ing\build\45b3b9a1aa73fdb3ff32afdc2b2ddc0affca7928\Ruby.vcxproj] NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.29.30133\bi n\HostX64\x64\cl.EXE"' : return code '0x2' [C:\Users\julien\.conan\data\openstudio_ruby\2.7.2\nrel\testing\build\45b3b9 a1aa73fdb3ff32afdc2b2ddc0affca7928\Ruby.vcxproj] ```

This reverts commit bfafa6e.

cf ruby/ruby@b091889#diff-60e6f0aa6e190c54f6cb0ac74148ced3cf0ef8703e1c3824a07a82dc54654953R13335

This is an admission of my failure to get Conan Package Tools to pass CONAN_BASE_BUILD_PROFILE to docker context

m

On conanio/gcc7, pthread_atfork is in pthread, while on ubuntu 20.04 it's in libc. I also tried to patch ruby (cf a7e0297) to reorder lcrypt and lpthread (so that lpthread is at the end) but that didn't work either

debugged the situation in docker conanio/gcc7

jmarrec · 2022-04-01T16:07:26Z

Holy crap, the linking command is so superbly wrong... I'm always amazed by the fact that ruby build system actually works in some cases...

ORIGINAL LINK FLAGS = -Wl,-rpath,/home/conan/.conan/data/openstudio_ruby/2.7.2/jmarrec/testing/build/9f9bc9c7ba86f123c671ad0cd57bd027161f8634/Ruby-prefix/src/Ruby-install/lib
 -lruby-static -lz -lpthread -lrt -lrt -lgmp -ldl -lcrypt -lm -lz -lpthread
 -lrt -lrt -lgmp -ldl -lcrypt -lm -lm -lanl -lssl -lcrypto
 -lreadline -lncurses -lgdbm_compat -lgdbm -lz -ldl -lffi -lutil -lyaml -lanl -lssl -lcrypto 
 -lreadline -lncurses -lgdbm_compat -lgdbm -lz -ldl -lffi -lutil -lyaml

Here are the link flags that work with conanio/gcc7: Removing duplicates, moving -pthread (not -lpthread at the end)

NEW LINK FLAGS =  -Wl,-rpath,/home/conan/.conan/data/openstudio_ruby/2.7.2/jmarrec/testing/build/9f9bc9c7ba86f123c671ad0cd57bd027161f8634/Ruby-prefix/src/Ruby-install/lib
 -lruby-static -lz -lrt -lgmp -ldl -lcrypt -lm -lanl -lssl -lcrypto 
 -lreadline -lncurses -lgdbm_compat -lgdbm -lffi -lutil -lyaml -pthread

jmarrec · 2022-04-01T16:14:45Z

CMakeLists.txt

+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/Ruby.util.patch
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0002-patch-encoding-CP720.patch
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0003-YYUSE.patch ${EXTRA_PATCH}
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0004-GCC7_8_fix_link_flags.patch


New patch for GCC 7 and 8.... it sucks to have to do yet another weird config trick, but that's the way it is

jmarrec · 2022-04-01T16:14:59Z

CMakeLists.txt

  #file(DOWNLOAD https://iweb.dl.sourceforge.net/project/gnuwin32/bison/2.4.1/bison-2.4.1-dep.zip ${CMAKE_BINARY_DIR}/tools/bison-2.4.1-dep.zip)
  if(NOT EXISTS ${CMAKE_BINARY_DIR}/tools/sed-4.2.1-dep.zip)
-    file(DOWNLOAD https://iweb.dl.sourceforge.net/project/gnuwin32/sed/4.2.1/sed-4.2.1-dep.zip ${CMAKE_BINARY_DIR}/tools/sed-4.2.1-dep.zip)
+    file(DOWNLOAD  https://sourceforge.net/projects/gnuwin32/files/sed/4.2.1/sed-4.2.1-dep.zip ${CMAKE_BINARY_DIR}/tools/sed-4.2.1-dep.zip)


sourceforge endpoint changed

jmarrec · 2022-04-01T16:15:28Z

conanfile.py

-        self.requires("openssl/1.1.0l") # fails with 1.1.1h https://github.com/openssl/openssl/issues/3884`
+        # Doesn't work with 3.x.
+        # Doesn't work on gcc 7 and 8 with 1.1.1n
+        self.requires("openssl/1.1.1n")


bump all deps

jmarrec · 2022-04-01T16:15:51Z

patches/0003-YYUSE.patch

+ RUBY_FUNC_EXPORTED size_t
+ rb_yytnamerr(struct parser_params *p, char *yyres, const char *yystr)
+ {
+-    YYUSE(p);


Patch to Remove YYUSE (which is now private in bison 3.7.5+)

cf ruby/ruby@b091889#diff-60e6f0aa6e190c54f6cb0ac74148ced3cf0ef8703e1c3824a07a82dc54654953R13335

jmarrec · 2022-04-01T16:16:06Z

CMakeLists.txt

+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/Ruby.patch
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/Ruby.util.patch
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0002-patch-encoding-CP720.patch
+      && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0003-YYUSE.patch ${EXTRA_PATCH}


new patch for all...

jmarrec · 2022-04-01T16:16:19Z

CMakeLists.txt

               && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/Ruby.win.272.patch  # && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/Ruby.nodynamic.patch
               && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0001-patch-to-support-version-ranges-for-MSVC.patch
               && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0002-patch-encoding-CP720.patch
+               && ${PATCH_EXE} -p1 < ${CMAKE_SOURCE_DIR}/patches/0003-YYUSE.patch


including windows

jmarrec · 2022-04-01T16:17:02Z

patches/0004-GCC7_8_fix_link_flags.patch

+		$(ECHO) NEW LINK FLAGS = -Wl,-rpath,$(libdir) -lruby-static -lz -lrt -lgmp -ldl -lcrypt -lm -lanl -lssl -lcrypto -lreadline -lncurses -lgdbm_compat -lgdbm -lffi -lutil -lyaml -pthread
+		$(Q) $(PURIFY) $(CC) $(LDFLAGS) $(XLDFLAGS) $(MAINOBJ) $(EXTOBJS) -Wl,-rpath,$(libdir) -lruby-static -lz -lrt -lgmp -ldl -lcrypt -lm -lanl -lssl -lcrypto -lreadline -lncurses -lgdbm_compat -lgdbm -lffi -lutil -lyaml -pthread $(OUTFLAG)$@


Patch for the link flags...

jmarrec · 2022-04-01T16:17:49Z

.github/workflows/build.yml


          echo "=========== MAKEFILE ================"
-          find ~/.conan/data/openstudio_ruby/2.7.2/nrel/testing/build -name "Makefile"|while read fname; do
+          find ~/.conan/data/openstudio_ruby/2.7.2/$CONAN_USERNAME/testing/build -name "Makefile"|while read fname; do


make debug work on forks too

…nce april 1

….12 as well conan-io/conan-center-index#10105

jmarrec · 2022-05-23T23:09:34Z

Latest run on my fork: https://github.com/jmarrec/conan-openstudio-ruby/runs/6563992930?check_suite_focus=true

jmarrec · 2022-05-23T23:32:11Z

.github/workflows/build.yml

      fail-fast: false
      matrix:
-        build_name: [GCC-7, GCC-8, GCC-9, GCC-10, GCC-11, Apple-Clang 11, Apple-Clang 12, MSVC-2022, MSVC-2019, MSVC-2017]
+        build_name: [GCC-7, GCC-8, GCC-9, GCC-10, GCC-11, Apple-Clang 11, Apple-Clang 12, MSVC-2022, MSVC-2019]


dropping MSVC 2017 (this will be required on OS side if we want to support 2022 anyways)

Good with me. We need to move to 2022 soon

jmarrec · 2022-05-23T23:32:58Z

build.py

+    builder.update_build_if(
+        lambda build: True,
+        new_settings={
+            "bison:build_type": "Release",
+            "ruby_installer:build_type": "Release",
+            "m4:build_type": "Release",
+        },
+    )


I had tried to pass a specific build profile to the docker but failed, cf revert in cd54db9

jmarrec · 2022-05-24T00:21:45Z

Ok this is ready to go: https://github.com/jmarrec/conan-openstudio-ruby/runs/6564537150?check_suite_focus=true

I have tested locally that I can build OpenStudio via NatLabRockies/OpenStudio#4596

…merged

jmarrec added 12 commits April 1, 2022 12:07

Bump deps

5504043

Use openssl 1.1.1m (Ruby 2.7.2 official uses 2.1.2.. but conan only h…

8c5efee

…as 1.x and 3.x, 3.x does not work) ruby/openssl#369

Do add conancenter remote as well

c17df6e

conancenter still isn't picked up What the hell? Debug why it's not creating SWIGRubyRuntime.hxx Missing dependency chain in windows Re-enable other platforms

make debug work for forks

e4816e0

Revert "put back bison. I'm getting an unrsolved symbol on windows"

417f999

This reverts commit bfafa6e.

Patch to Remove YYUSE (which is now private in bison 3.7.5+)

af55199

cf ruby/ruby@b091889#diff-60e6f0aa6e190c54f6cb0ac74148ced3cf0ef8703e1c3824a07a82dc54654953R13335

Sourceforge endpoint changed.

840c688

Generate and use a build profile in build.py...

cf8c309

This is an admission of my failure to get Conan Package Tools to pass CONAN_BASE_BUILD_PROFILE to docker context

Try again to work around it

43efba4

bump deps again: try openssl 1.1.1n and readline 8.1.2

095d2aa

m

Can't seem to shake the openssl failure on GCC 7 and 8

ae591ae

On conanio/gcc7, pthread_atfork is in pthread, while on ubuntu 20.04 it's in libc. I also tried to patch ruby (cf a7e0297) to reorder lcrypt and lpthread (so that lpthread is at the end) but that didn't work either

jmarrec force-pushed the develop branch from 5f75d5d to ae591ae Compare April 1, 2022 10:09

Patch the Makefile.in template to pass the right flags to link ruby

3582105

debugged the situation in docker conanio/gcc7

jmarrec requested a review from tijcolem April 1, 2022 16:13

jmarrec self-assigned this Apr 1, 2022

jmarrec added the enhancement New feature or request label Apr 1, 2022

jmarrec commented Apr 1, 2022

View reviewed changes

jmarrec added a commit to jmarrec/OpenStudio that referenced this pull request Apr 1, 2022

Bump conan openstudio ruby to NatLabRockies/conan-openstudio-ruby#39

4c39060

Use zlib/1.2.12

aa27ba6

jmarrec added a commit to jmarrec/OpenStudio that referenced this pull request Apr 4, 2022

Bump conan openstudio ruby to NatLabRockies/conan-openstudio-ruby#39

5f65193

jmarrec added 4 commits April 4, 2022 20:57

Update test_package to use minizip/1.2.12 to match zlib/1.2.12

c4c3ba7

pcre is creating a conflict with zlib on mac and msvc

4d7de4a

Remove MSVC 2017, since windows-2016 runner is deleted form Github si…

ad32198

…nce april 1

Remove workaround: I patched pcre/8.45 upstream to depend on zlib/1.2…

9a57670

….12 as well conan-io/conan-center-index#10105

jmarrec force-pushed the develop branch from 4114cb9 to 76cb185 Compare April 5, 2022 11:18

try to cross build for armv8, I suppose it won't work (?)

2f9af7c

jmarrec force-pushed the develop branch from 76cb185 to 2f9af7c Compare May 23, 2022 21:13

Merge branch 'fix_CI' into bump_deps

c79a619

jmarrec added 2 commits May 23, 2022 23:57

Revert: try to cross build for armv8, I suppose it won't work (?)

f7097e1

bump openssl to 1.1.1o

d6f43c1

jmarrec force-pushed the develop branch from c7748e1 to d6f43c1 Compare May 23, 2022 22:15

Merge remote-tracking branch 'origin/develop' into develop

eff7cba

Final cleanup hopefully

cd54db9

jmarrec commented May 23, 2022

View reviewed changes

jmarrec mentioned this pull request May 24, 2022

Bump conan dependencies (which will allow MSVC 2022 and C++20) and clang 13.1+ fixes NatLabRockies/OpenStudio#4596

Merged

19 tasks

tijcolem approved these changes May 24, 2022

View reviewed changes

tijcolem merged commit 3bd0590 into NatLabRockies:develop May 24, 2022

jmarrec deleted the develop branch June 13, 2022 14:52

jmarrec added a commit to NatLabRockies/OpenStudio that referenced this pull request Jun 13, 2022

Use nrel/testing now that NatLabRockies/conan-openstudio-ruby#39 was …

541d8dc

…merged

		+ $(ECHO) NEW LINK FLAGS = -Wl,-rpath,$(libdir) -lruby-static -lz -lrt -lgmp -ldl -lcrypt -lm -lanl -lssl -lcrypto -lreadline -lncurses -lgdbm_compat -lgdbm -lffi -lutil -lyaml -pthread
		+ $(Q) $(PURIFY) $(CC) $(LDFLAGS) $(XLDFLAGS) $(MAINOBJ) $(EXTOBJS) -Wl,-rpath,$(libdir) -lruby-static -lz -lrt -lgmp -ldl -lcrypt -lm -lanl -lssl -lcrypto -lreadline -lncurses -lgdbm_compat -lgdbm -lffi -lutil -lyaml -pthread $(OUTFLAG)$@

Conversation

jmarrec commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jmarrec commented Apr 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmarrec commented May 23, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jmarrec commented May 24, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jmarrec commented Apr 1, 2022 •

edited

Loading

jmarrec commented Apr 1, 2022 •

edited

Loading