Skip to content
This repository has been archived by the owner on Apr 22, 2023. It is now read-only.

Node.js compile fails in Amazon AMI x64 - snapshot.cc - pure functional method call #2912

Closed
brandwe opened this issue Mar 12, 2012 · 45 comments
Labels

Comments

@brandwe
Copy link

brandwe commented Mar 12, 2012

Version 0.7.5

LINK(target) /usr/local/src/node/out/Release/mksnapshot
LINK(target) /usr/local/src/node/out/Release/mksnapshot: Finished
ACTION v8_snapshot_run_mksnapshot /usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc
pure virtual method called
terminate called without an active exception
/bin/sh: line 1: 23819 Aborted "/usr/local/src/node/out/Release/mksnapshot" --log-snapshot-positions --logfile "/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.log" "/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc"
make[1]: *** [/usr/local/src/node/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 134
make[1]: Leaving directory `/usr/local/src/node/out'

@brandwe
Copy link
Author

brandwe commented Mar 12, 2012

Contents of log file:

code-creation,Stub,0x270d63206000,351,"JSEntryStub"
code-creation,Stub,0x270d63206160,351,"JSConstructEntryStub"
code-creation,Stub,0x270d632062c0,762,"CEntryStub"
code-creation,Stub,0x270d632065c0,1113,"CEntryStub"

@bnoordhuis
Copy link
Member

Looks like a V8 issue. You can disable snapshots with ./configure --without-snapshot. If you want to debug the issue, attach a debugger to mksnapshot and inspect the call stack. make BUILDTYPE=Debug compiles a debug build.

@brandwe
Copy link
Author

brandwe commented Mar 14, 2012

Attaching gdb now. I can get a clean compile in v8 standalone so it'll be interesting to see what's happening in snapshot in v8.

@ptejada
Copy link

ptejada commented Apr 4, 2012

After ./configure --without-snapshot i run make install and here is the output

make -C out BUILDTYPE=Release
make[1]: Entering directory /root/node-v0.7.6/out'
make[1]: Nothing to be done forall'.
make[1]: Leaving directory `/root/node-v0.7.6/out'
ln -fs out/Release/node node
out/Release/node tools/installer.js ./config.gypi install
pure virtual method called
terminate called without an active exception
make: *** [install] Aborted

Below is the output of g++ -v

Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.4.6 20110731 (Red Hat 4.4.6-3) (GCC)

@ghost
Copy link

ghost commented Apr 12, 2012

I'm seeing the same results as ptejada - any thoughts here?

@bnoordhuis
Copy link
Member

@spack8rivers: Try hooking up the faulting process (mksnapshot, node) to gdb and check where the method call comes from. Please post the backtrace here.

@quexer
Copy link

quexer commented Apr 14, 2012

same issue on my box.

#uname -a

Linux foo 2.6.32-71.el6.x86_64 #1 SMP Wed Sep 1 01:33:01 EDT 2010 x86_64 x86_64 x86_64 GNU/Linux

@bnoordhuis
Copy link
Member

People, I need a little more to go on. Can you try this?

$ touch empty.js
$ gdb --args node empty.js
> run
# wait for "pure virtual method called" exception
> backtrace full

Please post the backtrace here.

@ptejada
Copy link

ptejada commented Apr 14, 2012

Here is my output, do note i ran it in CentOS 6 x64

Starting program: /home/ptejada/webapps/node/src/node empty.js
[Thread debugging using libthread_db enabled]
pure virtual method called
terminate called without an active exception

Program received signal SIGABRT, Aborted.
0x00007ffff6cb7885 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
(gdb) backtrace full
#0  0x00007ffff6cb7885 in raise () from /lib64/libc.so.6
No symbol table info available.
#1  0x00007ffff6cb9065 in abort () from /lib64/libc.so.6
No symbol table info available.
#2  0x00007ffff7789a7d in __gnu_cxx::__verbose_terminate_handler() ()
   from /usr/lib64/libstdc++.so.6
No symbol table info available.
#3  0x00007ffff7787c06 in ?? () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#4  0x00007ffff7787c33 in std::terminate() () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#5  0x00007ffff778855f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
No symbol table info available.
#6  0x000000000095aac4 in v8::internal::CodeStub::FindCodeInCache(v8::internal::Code**) ()
No symbol table info available.
#7  0x000000000095afb8 in v8::internal::CodeStub::GetCode() ()
No symbol table info available.
#8  0x00000000008cb1f1 in v8::internal::CodeStub::GenerateStubsAheadOfTime() ()

@bnoordhuis
Copy link
Member

@ptejada: Thanks. This looks like it could be a number of things but it's probably a V8 issue.

Can you check if it happens with debug builds too? You can build one with make -C out BUILDTYPE=Debug, then run gdb --args out/Debug/node empty.js. Please post the backtrace, it should have line numbers.

@ptejada
Copy link

ptejada commented Apr 15, 2012

There was no stack for the backtrace, here is the console output:

(gdb) run
Starting program: /home/ptejada/webapps/node/src/out/Debug/node empty.js
[Thread debugging using libthread_db enabled]
[New Thread 0x7ffff7fe4700 (LWP 11211)]
[Thread 0x7ffff7fe4700 (LWP 11211) exited]

Program exited normally.
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.47.el6_2.9.x86_64 libgcc-4.4.6-3.el6.x86_64 libstdc++-4.4.6-3.el6.x86_64
(gdb) backtrace
No stack.
(gdb) backtrace full
No stack.
(gdb)

@bnoordhuis
Copy link
Member

Interesting. So debug builds work but release builds don't. That suggests we're looking at either a compiler bug or a build type-dependent code path in V8.

Can I persuade you to try it with the V8 shell? Here is how you build it:

$ git clone git://github.com/v8/v8.git
$ cd v8/
$ git checkout -b bleeding_edge -t origin/bleeding_edge
$ scons --jobs=<num-cores> arch=x64 mode=debug,release sample=shell
# wait while it compiles
$ touch empty.js
$ gdb --args ./shell_g empty.js
# do the run/backtrace thing here
$ gdb --args ./shell empty.js
# ditto

Testing the 32 bits build might be worthwhile too, just replace arch=x64 with arch=ia32.

@hammer2j2
Copy link

Linux hail.liaoningbeauties.com 2.6.32-131.0.15.el6.x86_64 #1 SMP Tue May 10 15:42:40 EDT 2011 x86_64 x86_64 x86_64 GNU/Linux

-bash-4.1$ gdb --args node t.js
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-48.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /home/alibaba/joyent-node-93cefab/node...done.
(gdb) run
Starting program: /home/alibaba/joyent-node-93cefab/node t.js
[Thread debugging using libthread_db enabled]
pure virtual method called
terminate called without an active exception

Program received signal SIGABRT, Aborted.
0x0000003e6e232a45 in raise () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install glibc-2.12-1.25.el6.x86_64 libgcc-4.4.5-6.el6.x86_64 libstdc++-4.4.5-6.el6.x86_64
(gdb) bt
#0 0x0000003e6e232a45 in raise () from /lib64/libc.so.6
#1 0x0000003e6e234225 in abort () from /lib64/libc.so.6
#2 0x0000003e7aebea7d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib64/libstdc++.so.6
#3 0x0000003e7aebcc06 in ?? () from /usr/lib64/libstdc++.so.6
#4 0x0000003e7aebcc33 in std::terminate() () from /usr/lib64/libstdc++.so.6
#5 0x0000003e7aebd55f in __cxa_pure_virtual () from /usr/lib64/libstdc++.so.6
#6 0x0000000000971dc4 in v8::internal::CodeStub::FindCodeInCache(v8::internal::Code**) ()
#7 0x00000000009722b8 in v8::internal::CodeStub::GetCode() ()
#8 0x00000000008e2501 in v8::internal::CodeStub::GenerateStubsAheadOfTime() ()
#9 0x0000000000747623 in v8::internal::Heap::CreateFixedStubs() ()
#10 0x0000000000751594 in v8::internal::Heap::CreateInitialObjects() ()
#11 0x000000000075e284 in v8::internal::Heap::SetUp(bool) ()
#12 0x00000000007b8bc0 in v8::internal::Isolate::Init(v8::internal::Deserializer*) ()
#13 0x000000000057ef4b in node::Start(int, char**) ()
#14 0x0000003e6e21ec9d in __libc_start_main () from /lib64/libc.so.6
#15 0x0000000000575d8d in _start ()

@hammer2j2
Copy link

Right, the debug compile doesn't crash on my hello world script.

out/Debug/node t.js
hello world.

@hammer2j2
Copy link

I ran the steps provided in comment 5139451 by Ben Noordhuis, and there were no errors no backtrace.

@bnoordhuis
Copy link
Member

@hammer2j2: Does a release build work if you disable snapshots (--without-snapshot)? Does the V8 shell work?

@hammer2j2
Copy link

I did, disable snapshot. It's the only way the release build will succeed. There's I think several ppl reporting similar issue there. thanks.

@bnoordhuis
Copy link
Member

Can someone test the V8 shell? I need to narrow down the problem and I cannot reproduce it. You can try the bundled V8:

$ cd deps/v8
$ scons --jobs=<num-cores> arch=x64 mode=debug,release sample=shell
$ touch empty.js
$ ./shell_g empty.js
$ ./shell empty.js

If either shell or shell_g crashes, please post the backtrace.

@hammer2j2
Copy link

Regarding v8 shell: I ran the steps provided in comment 5139451. There were no errors no backtrace.

@bnoordhuis
Copy link
Member

Thanks, that's useful info. Can you try it with the bundled V8? If that one does crash, it's been fixed upstream.

@hammer2j2
Copy link

$ cd deps/v8
$ scons --jobs= arch=x64 mode=debug,release sample=shell
$ touch empty.js
$ ./shell_g empty.js
$ ./shell empty.js

Runs w/o error.

@bnoordhuis
Copy link
Member

Okay, that's kind of unfortunate: it means it's not a straightforward V8 bug and I'll have to do some digging. Can I get shell access on a machine that exhibits the behavior?

@hammer2j2
Copy link

bnoordhuis - I will work on getting a hosted environment setup for this by Monday.

Thanks.

@ptejada
Copy link

ptejada commented Apr 19, 2012

I have a clean test account with access to bash on a centOS 6 x64

How can i get the accout info to you?

@bnoordhuis
Copy link
Member

@ptejada: Email me the account and machine details (address is in my profile) and I'll email back my public key. vim, git and gdb is all I need. No rush, I can't look at it until later tonight anyway. :-)

EDIT: I forgot, GNU screen would be nice too.

@ptejada
Copy link

ptejada commented Apr 19, 2012

@bnoordhuis There was actually no public email in your profile, so i sent it to the email on your site. Info@yoursite

@bnoordhuis
Copy link
Member

Fixed in c21c51a, thanks @ptejada.

Turns out it's an obscure gcc bug. It didn't show up in standalone builds of V8 because it doesn't (by default) compile with the -fno-strict-aliasing flag that triggers the bug. I'll forward the patch to the V8 team.

@bnoordhuis
Copy link
Member

@ptejada
Copy link

ptejada commented Apr 19, 2012

@bnoordhuis sure no problem

@bnoordhuis bnoordhuis reopened this May 2, 2012
@bnoordhuis
Copy link
Member

Reopening. I've reverted the commit in 6b426a2, it was making mksnapshot segfault on sunos. Compiler bugs, you just can't win...

@bwalding
Copy link

bwalding commented May 3, 2012

What is the impact of compiling with --standalone?

We are seeing the same thing across a range of 0.7 versions on 64 bit EC2 hardware (m1.large) running Fedora core:

$ gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info
--with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared 
--enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit 
--disable-libunwind-exceptions --enable-gnu-unique-object 
--enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi 
--with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile 
--enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar 
--disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 
--build=x86_64-redhat-linux

Thread model: posix
gcc version 4.4.5 20101112 (Red Hat 4.4.5-2) (GCC) 

$ uname -a
Linux localhost 2.6.34.7-66.fc13.x86_64 #1 SMP Wed Dec 15 07:04:30 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

Our builds can be seen here:
https://cbruby.ci.cloudbees.com/job/build-all-nodes/

@bnoordhuis
Copy link
Member

What is the impact of compiling with --standalone?

Standalone as in linking to an external libv8? It usually works but there are no guarantees - V8's API and ABI aren't stable.

If you mean --without-snapshot, there is no runtime performance overhead, only a slightly longer startup time.

@bwalding
Copy link

bwalding commented May 8, 2012

@bnoordhuis - Thanks for the info - unfortunately compiling using "--without-snapshot" did not improve the situation.

Still getting errors building >= 0.7.6 on x86_64

out/Release/node tools/installer.js install 
pure virtual method called
terminate called without an active exception
make: *** [install] Aborted

@bnoordhuis
Copy link
Member

@bwalding You can apply the patch manually but I recommend you upgrade gcc.

@sjonnet19
Copy link

I am receiving this error now with 0.8.2. I wasn't previously with 0.8.0 or 0.8.1 which is odd

pure virtual method called
terminate called without an active exception
/bin/sh: line 1: 32557 Aborted "/usr/local/n/node-v0.8.2/out/Release/mksnapshot" --log-snapshot-positions --logfile "/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.log" "/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc"
make[1]: *** [/usr/local/n/node-v0.8.2/out/Release/obj.target/v8_snapshot/geni/snapshot.cc] Error 134
make[1]: Leaving directory `/usr/local/n/node-v0.8.2/out'
make: *** [node] Error 2

@skovalyov
Copy link

Have the same with 0.8.2

@sjonnet19
Copy link

Should I open a new ticket for this?

@bnoordhuis
Copy link
Member

@sjonnet19 No need. Compile with -O2 for now (i.e. make CFLAGS+=-O2 CXXFLAGS+=-O2). That's going to be the default in v0.8.3 anyway.

If you have some time to kill, I'd be curious to know if compiling with either -fno-inline-functions or -fno-function-sections solves the issue for you.

@skovalyov
Copy link

Thanks! Advice from #3675 works fine.

@sjonnet19
Copy link

Thanks that works I am now dynamically patching n

@bnoordhuis
Copy link
Member

Guys, can one of you test commit b40f813? -O3 but no -ffunction-sections.

@sjonnet19
Copy link

No luck failed

@bnoordhuis
Copy link
Member

@sjonnet19 What if you apply the patch and compile with make CFLAGS+=-fno-strict-aliasing CXXFLAGS+=-fno-strict-aliasing?

@sjonnet19
Copy link

No luck either

@bnoordhuis
Copy link
Member

This should be resolved now (where now = 0.8.4). I haven't had bug reports in a while so I'm going to assume that it is.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

8 participants