citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

richardlau · 2018-04-27T21:28:44Z

Refs: nodejs/node#20137 (comment)

The stress job appears not to be using the compiler selector script added by #1240 so it's using the wrong level of gcc on AIX for master/v10.x.

e.g., https://ci.nodejs.org/job/node-stress-single-test/1825/nodes=aix61-ppc64/consoleFull

16:38:06 WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=g++)

The text was updated successfully, but these errors were encountered:

addaleax · 2018-04-28T18:17:18Z

This apparently goes for CITGM as well: https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1390/nodes=rhel72-s390x/console

Work around nodejs/build#1254, which effectively breaks stress test CI and CITGM, by avoiding `std::make_unique` for now. This workaround should be reverted once that issue is resolved. Refs: nodejs/build#1254

Work around nodejs/build#1254, which effectively breaks stress test CI and CITGM, by avoiding `std::make_unique` for now. This workaround should be reverted once that issue is resolved. Refs: nodejs/build#1254 PR-URL: #20386 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Matheus Marchini <matheus@sthima.com>

mhdawson · 2018-04-30T15:44:05Z

Fixing the stress job.

@gdams can you update the CITGM jobs, they need to use

rm -rf build
git clone https://github.com/nodejs/build.git
. ./build/jenkins/scripts/select-compiler.sh

For all platforms (selection is currently done on PPC, s390 and AIX, but will be on others in the future). In addition for AIX if there is a CC= definition it should be removed as that is now set by the selection script.

mhdawson · 2018-05-01T00:00:19Z

Stress job fixed.

Work around nodejs/build#1254, which effectively breaks stress test CI and CITGM, by avoiding `std::make_unique` for now. This workaround should be reverted once that issue is resolved. Refs: nodejs/build#1254 PR-URL: #20386 Reviewed-By: Tiancheng "Timothy" Gu <timothygu99@gmail.com> Reviewed-By: Matheus Marchini <matheus@sthima.com>

richardlau · 2018-05-22T21:04:34Z

Ping @gdams re. the CitGM jobs ^

Note that this is affecting the v10.2.0 PR: nodejs/node#20724 (comment)

mhdawson · 2018-05-23T04:49:51Z

@gdams can you prioritize updating the Citgm jobs?

gdams · 2018-05-23T10:13:20Z

I have added this script to the citgm-smoker job

MylesBorins · 2018-05-23T15:57:32Z

thanks all! Can this be considered fixed now?

mhdawson · 2018-05-23T19:04:23Z

@gdams did you test with a citgm run? I think we want to do that and then we can close.

The key thing is to make sure that the job selects the right compiler based on the Node.js version. If the job builds node.js from source then it should do the right thing. For jobs that use existing binaries it needs an extra step that I added to the job for the tests for the node-addon-api.

richardlau · 2018-05-23T19:15:46Z

I think the CitGM jobs may need more work. From the latest v10.2.0 CitGM runs:

rhel72-s390x

+ . ./build/jenkins/scripts/select-compiler.sh
++ '[' '' '!=' DONT ']'
++ case $NODE_NAME in
++ SELECT_ARCH=S390X
++ '[' S390X = PPC64LE ']'
++ '[' S390X = S390X ']'
++ export COMPILER_LEVEL=
++ COMPILER_LEVEL=
+++ python tools/getnodeversion.py
++ NODE_VERSION=10.2.0
+++ echo 10.2.0
+++ cut -d . -f 1
++ NODE_MAJOR_VERSION=10
++ echo 'Setting compiler for Node version 10 on s390x'
Setting compiler for Node version 10 on s390x
++ '[' 10 -gt 9 ']'
++ export PATH=/data/gcc-4.9/bin:/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ PATH=/data/gcc-4.9/bin:/usr/lib/ccache:/usr/lib64/ccache:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
++ export LD_LIBRARY_PATH=/data/gcc-4.9/lib64:
++ LD_LIBRARY_PATH=/data/gcc-4.9/lib64:
++ export COMPILER_LEVEL=-4.9
++ COMPILER_LEVEL=-4.9
++ export 'CC=ccache gcc-4.9'
++ CC='ccache gcc-4.9'
++ export 'CXX=ccache g++-4.9'
++ CXX='ccache g++-4.9'
++ export LINK=g++-4.9
++ LINK=g++-4.9
++ echo 'Compiler set to -4.9'
Compiler set to -4.9
...
+ node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)

https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/nodes=rhel72-s390x/consoleFull

aix61-ppc64

+ . ./build/jenkins/scripts/select-compiler.sh
++ '[' '' '!=' DONT ']'
++ case $NODE_NAME in
++ SELECT_ARCH=AIXPPC
++ '[' AIXPPC = PPC64LE ']'
++ '[' AIXPPC = S390X ']'
++ '[' AIXPPC = AIXPPC ']'
+++ python tools/getnodeversion.py
++ NODE_VERSION=10.2.0
+++ echo 10.2.0
+++ cut -d . -f 1
++ NODE_MAJOR_VERSION=10
++ echo 'Setting compiler for Node version 10 on AIX'
Setting compiler for Node version 10 on AIX
++ '[' 10 -gt 9 ']'
++ export LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
++ LIBPATH=/home/iojs/gmake/opt/freeware/lib:/home/iojs/gcc-6.3.0-1/opt/freeware/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/pthread/ppc64:/home/iojs/gcc-6.3.0-1/opt/freeware/lib
++ export PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
++ PATH=/home/iojs/gcc-6.3.0-1/opt/freeware/bin:/opt/freeware/bin/ccache:/home/iojs/jdk8/jre/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java6/jre/bin:/usr/java6/bin
+++ which gcc
+++ which g++
+++ which g++
++ export 'CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc' 'CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++' 'CXX_host=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ CC='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc'
++ CXX='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ CXX_host='ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++'
++ echo 'Compiler set to 6.3'
Compiler set to 6.3
+ CONFIG_FLAGS='--with-intl=small-icu --download=all --download-path=/home/iojs/node-icu/'
+ mkdir -p /home/iojs/node-icu/
+ [[ 10.2.0 =~ ^0\.10 ]]
+ rm -rf /home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker
+ mkdir /home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker
++ uname -s
+ '[' AIX = AIX ']'
+ MAKE=gmake
+ unset LIBPATH
+ echo

+ python ./configure --prefix=/home/iojs/build/workspace/citgm-smoker/nodes/aix61-ppc64/smoker --with-intl=small-icu --download=all --download-path=/home/iojs/node-icu/
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1plus because of the following errors:
	0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
	0509-152   Member libmpc.so.3 is not found in archive 
exec(): 0509-036 Cannot load program /home/iojs/gcc-6.3.0-1/opt/freeware/bin/../libexec/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/cc1 because of the following errors:
	0509-150   Dependent module /opt/freeware/lib/libmpc.a(libmpc.so.3) could not be loaded.
	0509-152   Member libmpc.so.3 is not found in archive 
WARNING: C++ compiler too old, need g++ 4.9.4 or clang++ 3.4.2 (CXX=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/g++)
WARNING: C compiler too old, need gcc 4.2 or clang 3.2 (CC=ccache /home/iojs/gcc-6.3.0-1/opt/freeware/bin/gcc)
Traceback (most recent call last):
  File "./configure", line 1475, in <module>
    configure_node(output)
  File "./configure", line 886, in configure_node
    host_arch = host_arch_win() if os.name == 'nt' else host_arch_cc()
  File "./configure", line 798, in host_arch_cc
    k = cc_macros('gcc')
  File "./configure", line 755, in cc_macros
    p.stdin.write('\n')
IOError: [Errno 32] Broken pipe

https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/nodes=aix61-ppc64/console

We shouldn't be attempting to build Node.js 10 on AIX 6.1 as the minimum supported level is 7.1 TL04: https://github.com/nodejs/node/blob/v10.0.0/BUILDING.md#supported-platforms
(aix61-ppc64 is the only AIX axis in https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1435/).

mhdawson · 2018-05-23T19:19:58Z

@richardlau, while in the perfect world you are correct for Node 10.x the reality is that we don't have the hw at this point and building on 6.1 is what we need to do. We made the minimum level 7.1 to that we'd have the flexibility of moving up to that but we are not planning to build the infrastructure in the near future so we SHOULD continue to build 10.x on AIX 6.1 for now.

richardlau · 2018-05-23T19:23:27Z

@mhdawson ack. The CitGM job is failing on AIX with the compiler script though.

mhdawson · 2018-05-23T19:26:45Z

@gdams the problem with the runs shown for 9.x and 10.x above is that previously CC was set in the configure step. When I updated the other jobs I had to remove that as CC is now set in the compiler selection script. I'm guessing that if the original set of CC is still there it is overwriting what the script had put in place.

mhdawson · 2018-05-29T00:12:52Z

@gdams do you have a timeline for when you might be able to look at this?

mhdawson · 2018-05-29T00:16:52Z

One problem was that the LIBPATH was being unset after the selection, moved it to be after and we'll see if that helps. @gdams I only changed this job https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/ so others probably need the same change.

mhdawson · 2018-05-29T00:44:04Z

Looks to have run much further, but still compiling.

gdams · 2018-05-29T10:09:04Z

so I can see the following failure on the aix jobs:

+ . ./build/jenkins/scripts/select-compiler.sh
+ [  != DONT ]
+ SELECT_ARCH=AIXPPC
+ [ AIXPPC = PPC64LE ]
+ [ AIXPPC = S390X ]
+ [ AIXPPC = AIXPPC ]
+ + python tools/getnodeversion.py
python: can't open file 'tools/getnodeversion.py': [Errno 2] No such file or directory
NODE_VERSION=

I'll take a look at what's going wrong

mhdawson · 2018-05-29T15:58:03Z

I fixed it up and seems to be ok now, had to do with the current directory when the selection was being done.

mhdawson · 2018-05-29T16:00:01Z

It now ran ok I believe but like other platforms there are failures. https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/1445/nodes=aix61-ppc64/console

@gdams can you see if these match the other platforms. If so I think we can close this issue.

gdams · 2018-05-30T23:07:13Z

I'm still seeing this failure on s390x:

+ node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)

https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/nodes=rhel72-s390x/1446/console

richardlau · 2018-11-29T02:39:23Z

I'm still seeing this failure on s390x:
+ node -v
node: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by node)
https://ci.nodejs.org/view/Node.js-citgm/job/citgm-smoker/nodes=rhel72-s390x/1446/console

This was fixed by #1548.

addaleax mentioned this issue Apr 28, 2018

src: avoid std::make_unique nodejs/node#20386

Closed

2 tasks

richardlau mentioned this issue May 22, 2018

v10.2.0 proposal nodejs/node#20724

Merged

This was referenced May 22, 2018

"NodeTracing" domain introduction nodejs/node#20608

Merged

inspector: get rid of the make_unique nodejs/node#20895

Merged

maclover7 added the ci-public label May 22, 2018

richardlau changed the title ~~Stress job doesn't select compiler for AIX on v10.x/master~~ citgm/stress jobs do not select compiler for AIX on v10.x/master May 24, 2018

richardlau closed this as completed Nov 29, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

richardlau commented Apr 27, 2018

addaleax commented Apr 28, 2018

mhdawson commented Apr 30, 2018

mhdawson commented May 1, 2018

richardlau commented May 22, 2018

mhdawson commented May 23, 2018

gdams commented May 23, 2018

MylesBorins commented May 23, 2018

mhdawson commented May 23, 2018

richardlau commented May 23, 2018 •

edited

Loading

mhdawson commented May 23, 2018 •

edited

Loading

richardlau commented May 23, 2018

mhdawson commented May 23, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

gdams commented May 29, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

gdams commented May 30, 2018

richardlau commented Nov 29, 2018

citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

citgm/stress jobs do not select compiler for AIX on v10.x/master #1254

Comments

richardlau commented Apr 27, 2018

addaleax commented Apr 28, 2018

mhdawson commented Apr 30, 2018

mhdawson commented May 1, 2018

richardlau commented May 22, 2018

mhdawson commented May 23, 2018

gdams commented May 23, 2018

MylesBorins commented May 23, 2018

mhdawson commented May 23, 2018

richardlau commented May 23, 2018 • edited Loading

mhdawson commented May 23, 2018 • edited Loading

richardlau commented May 23, 2018

mhdawson commented May 23, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

gdams commented May 29, 2018

mhdawson commented May 29, 2018

mhdawson commented May 29, 2018

gdams commented May 30, 2018

richardlau commented Nov 29, 2018

richardlau commented May 23, 2018 •

edited

Loading

mhdawson commented May 23, 2018 •

edited

Loading