Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PPC: test-osuosl-ubuntu1404-ppc64_le-2 has strange linker issues #1815

Closed
refack opened this issue May 30, 2019 · 12 comments
Closed

PPC: test-osuosl-ubuntu1404-ppc64_le-2 has strange linker issues #1815

refack opened this issue May 30, 2019 · 12 comments

Comments

@refack
Copy link
Contributor

refack commented May 30, 2019

https://ci.nodejs.org/computer/test-osuosl-ubuntu1404-ppc64_le-2/ has been consistently failing with a linker error:

11:16:01 g++-6 -o /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/mksnapshot -pthread -rdynamic -m64 -m64 -Wl,--start-group /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/mksnapshot/deps/v8/src/snapshot/embedded-file-writer.o /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/mksnapshot/deps/v8/src/snapshot/mksnapshot.o /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_base.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_init.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_libbase.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_libplatform.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_nosnapshot.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/icu/libicui18n.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_libsampler.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/icu/libicuucx.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/icu/libicudata.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/icu/libicustubdata.a /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/tools/v8_gypfiles/libv8_initializers.a -ldl -lrt -latomic -Wl,--end-group
11:16:04 /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/v8_base/deps/v8/src/bootstrapper.o:(.toc1+0x218): undefined reference to v8::internal::StatisticsExtension::kSource' 11:16:04 /home/iojs/build/workspace/node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/v8_base/deps/v8/src/bootstrapper.o:(.toc1+0x228): undefined reference to vtable for v8::internal::StatisticsExtension'


I marked it offline for the time being.

![image](https://user-images.githubusercontent.com/96947/58643172-c6195900-82cc-11e9-8889-81f2e541e0ee.png)
@refack
Copy link
Contributor Author

refack commented May 30, 2019

/CC @mhdawson @sam-github

@mhdawson
Copy link
Member

mhdawson commented Jun 4, 2019

I've pinged @sam-github to make sure this is on his radar. We were both out last week at the Collab summit.

@sam-github
Copy link
Contributor

@miladfarca, you looked at this a bit? Did you have any ideas, or even manage to reproduce this?

@refack

This comment has been minimized.

@richardlau

This comment has been minimized.

@miladfarca
Copy link

@sam-github this is a separate issue I assume, the one I was looking at was related to the /test-worker-prof.js test case.

@refack

This comment has been minimized.

@sam-github
Copy link
Contributor

The history here is a bit confusing, because multiple issues are discussed. I hid @refack and @richardlau comments above that are related to #1820, just to make this easier to read.

@miladfarca, we have discussed this linker error before, in slack:

16:06:17   g++ -o /home/iojs/build/workspace/node-test-commit-plinux/nodes/centos7-ppcle/out/Release/obj.target/v8_base/deps/v8/src/extensions/statistics-extension.o ../deps/v8/src/extensions/statistics-extension.cc '-DV8_GYP_BUILD' '-DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64' '-DOPENSSL_THREADS' '-DV8_TARGET_ARCH_PPC' '-DV8_TARGET_ARCH_PPC64' '-
Milad Farazmand  12 days ago
The build that's failing isn't compiling this file, hence a missing symbol error, now look into why this file is not included when you are compiling your code (edited)
SAM ROBERTS  12 days ago
interesting. and odd :disappointed:. thanks for looking.
Milad Farazmand  12 days ago
no issues,  might be helpful to look into tools/v8_gypfiles/v8.gyp, every machine specific file that needs to be compiled for v8 is listed there

Luckily (?) the linker problems stopped over the weekend, and my build passed and was merged (by someone else): nodejs/node#27834

But, there seems to be a real issue.

Reviewing https://ci.nodejs.org/computer/test-osuosl-ubuntu1404-ppc64_le-2/builds, many failed with linker errors (though some with pty timeouts, as in #1820).

I'll go look at the .gyp file that Milad mentions in the workspace, and compare it to a clean build. Since the workers is offline, I should be able to redo the build in the workspace from ssh, and see if it fails to link for me, too.

@miladfarca
Copy link

miladfarca commented Jun 18, 2019

v8::internal::StatisticsExtension::kSource is defined in statistics-extension.cc under V8. Looking at the latest job under this machine (hash cfbb5bb) on May 30, the statistics-extension object file is not compiled properly, this is the output of nm:

$ nm out/Release/obj.target/v8_base/deps/v8/src/extensions/statistics-extension.o

nm: out/Release/obj.target/v8_base/deps/v8/src/extensions/statistics-extension.o: File truncated

I copied the directory, re configured and compiled it from scratch and it worked fine. Seems like jenkins was interrupted mid compilation and the truncated statistics-extension.o was used in the later test jobs and commits?

Are we clearing cache/workspace properly before/after each build?
Is it possible to clear the workspace manually and activate the machine and see if it happens again.

@sam-github
Copy link
Contributor

sam-github commented Jun 18, 2019

Its possible the workspace isn't cleared, but its also possible that a bad build got ccached, which would cause the issue to come back over and over.

I removed the .o file, cleared the ccache, and reenabled the machine.

iojs@test-osuosl-ubuntu1404-ppc64--le-2:~/build/workspace$ rm ./node-test-commit-plinux/nodes/ppcle-ubuntu1404/out/Release/obj.target/v8_base/deps/v8/src/extensions/statistics-extension.o

EDIT: I'll leave this open until some builds pass on this machine.

@sam-github
Copy link
Contributor

I just got a green build from the daily: https://ci.nodejs.org/computer/test-osuosl-ubuntu1404-ppc64_le-2/builds

Perhaps this issue is resolved?

@sam-github
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants