Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Node.js[5483]: ../src/node.cc:701:void node::ResetStdio(): Assertion `!(err != 0) || (err == -1 && (*__errno_location ()) == 1)' failed. #51519

Closed
TheRealRaeed opened this issue Jan 19, 2024 · 11 comments
Labels
stalled Issues and PRs that are stalled.

Comments

@TheRealRaeed
Copy link

TheRealRaeed commented Jan 19, 2024

Version

18.19.0

Platform

Linux localhost 5.4.0-90-generic #101-Ubuntu SMP Fri Oct 15 20:00:55 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Running nodejs with logging enabled

How often does it reproduce? Is there a required condition?

It reproduces periodically at weird intervals. Doesn't appear to have an actual condition

What is the expected behavior? Why is that the expected behavior?

The expected behavior is that it should be running without issues or at least posting some genuine error. This is expected because this error originates out of a NodeJS core lib, meaning we are dealing with some core error which is usually not expected.

What do you see instead?

Node.js v18.19.0
Node.js[5483]: ../src/node.cc:701:void node::ResetStdio(): Assertion `!(err != 0) || (err == -1 && (*__errno_location ()) == 1)' failed.
     1: 0xb95b60 node::Abort() [node]
     2: 0xb95bde  [node]
     3: 0xb4cdd4  [node]
     4: 0x7f6332fe58a7  [/lib/x86_64-linux-gnu/libc.so.6]
     5: 0x7f6332fe5a60 on_exit [/lib/x86_64-linux-gnu/libc.so.6]
     6: 0xad614a node::DefaultProcessExitHandler(node::Environment*, int) [node]
     7: 0xb1a3f3 node::Environment::Exit(int) [node]
     8: 0xb9909a node::errors::TriggerUncaughtException(v8::Isolate*, v8::Local<v8::Value>, v8::Local<v8::Message>, bool) [node]
     9: 0xb997d1  [node]
    10: 0xdcd3e0  [node]
    11: 0xdce91f v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [node]
    12: 0x170dfb9  [node]

Additional information

No response

@marco-ippolito
Copy link
Member

marco-ippolito commented Jan 19, 2024

Can you provide a minimal repro?
This is the assertion that fails

      // Normally we expect err == 0. But if macOS App Sandbox is enabled,
      // tcsetattr will fail with err == -1 and errno == EPERM.
      CHECK_IMPLIES(err != 0, err == -1 && errno == EPERM);

@TheRealRaeed
Copy link
Author

TheRealRaeed commented Jan 20, 2024

Try making a simple express app and running it in linux using this command
image

Then do some other simple linux commands and then log off the server using exit you will notice (if it happens) that it will go down, sometimes it takes a few days or hours, but if you visit the express site throughout the day, and use the linux server too, eventually this error comes up and you will notice the app has crashed.

@marco-ippolito
Copy link
Member

Without a reproducible snippet of code, it seems very hard to me to find out what is causing this

@RedYetiDev RedYetiDev added the stalled Issues and PRs that are stalled. label Aug 9, 2024
Copy link
Contributor

github-actions bot commented Aug 9, 2024

This issue/PR was marked as stalled, it will be automatically closed in 30 days. If it should remain open, please leave a comment explaining why it should remain open.

@RedYetiDev
Copy link
Member

Hi! I've marked this issue as stalled, as @marco-ippolito asked for a minimal reproduction back in January, and one is yet to be provided.

@RedYetiDev
Copy link
Member

(The stalled bot doesn't appear to be working, so I've manually closed. Feel free to reopen if this is no longer stalled)

@RedYetiDev RedYetiDev closed this as not planned Won't fix, can't repro, duplicate, stale Sep 24, 2024
@boutell
Copy link

boutell commented Oct 26, 2024

I just experienced this:

Node.js[843246]: ../src/node.cc:701:void node::ResetStdio(): Assertion `!(err != 0) || (err == -1 && (*__errno_location ()) == 1)' failed.
node --version
v18.20.4

My script is running for a long period of time (hours) and producing a lot of output on stdout and a small amount on stderr. It is not reading anything from stdin.

I don't really know how to get from here to a minimal repro because I have tons of nodejs scripts, and no others have ever done this. My script spends a lot of time in child_process.execSync calls, I don't know if that could be relevant or if it's entirely irrelevant (e.g. perhaps this part of src/node.cc is concerned entirely with the stdout of the main process itself, which is not involved, since I'm not passing interesting options like stdio: 'inherit').

For the last hour-plus before it fails, this script has done nothing but orchestrate a series cp.execSync calls to the AWS CLI, with no fancy options beyond { encoding: 'utf8' }. I do pay attention to the return value and parse it as JSON on each call. I can't see anything that would cause memory pressure because it works in small batches and retains nothing between batches. Stdout and stderr are both being piped to a file (at the bash shell level).

I've restarted it with much less output to stdout and will report on whether that seems to matter at all.

@boutell
Copy link

boutell commented Oct 26, 2024

Unfortunately this happens consistently even with much less output.

I'm going to try zero output just to see if it really has anything to do with normal use of standard output at all.

I neglected to mention I'm running Ubuntu 22.04 LTS.

@RedYetiDev
Copy link
Member

If you can provide a minimal reproduction, it'd be easier for someone to reproduce.

@boutell
Copy link

boutell commented Oct 26, 2024

I realize that. The difficulty is that it's probably dependent on specific responses I'm getting from an external service (AWS) and they contain information that isn't mine to share. But if I can nail it down and it's not something sensitive I will share it.

@lukas-mertens
Copy link

I just came here because we are experiencing the same / a similar issue in our pipeline from time to time. It seems like it appears whenever our build runner has high utilization. So it might have something to do with the available system resources which makes it hard to reproduce. It sometimes appears in our pipeline, sometimes it does not. We are running in qemu VMs with ubuntu-22.04 inside:

#  /nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node[16614]: void node::ResetStdio() at ../../src/node.cc:729
  #  Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1)

----- Native stack trace -----

 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 2: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 3: 0xbc6562 node::TearDownOncePerProcess() [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 4: 0xbcd2f0 node::Start(int, char**) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 5: 0x7fbc834b914e  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 6: 0x7fbc834b9209 __libc_start_main [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 7: 0xaeb455 _start [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 ELIFECYCLE  Command failed with exit code 134.

  #  Node.js[16557]: void node::ResetStdio() at ../../src/node.cc:729
  #  Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1)

----- Native stack trace -----

 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 2: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 3: 0x7f9e97f15b56  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 4: 0x7f9e97f15c8e  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 5: 0xb21029  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 6: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 7: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 8: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 9: 0xed09c8  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
10: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
11: 0x18dcdf6  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]

  #  Node.js[16557]: void node::ResetStdio() at ../../src/node.cc:729
  #  Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1)

----- Native stack trace -----

 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 2: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 3: 0xbc4ffc node::TrapWebAssemblyOrContinue(int, siginfo_t*, void*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 4: 0x7f9e97f135c0  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 5: 0x[154](https://github.com/PionixPro/summiteer-cloud/actions/runs/11857584699/job/33046303079#step:20:155)0450 v8::internal::tracing::TraceEventHelper::GetTracingController() [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 6: 0xff7ca5 v8::internal::Isolate::CaptureDetailedStackTrace(int, v8::StackTrace::StackTraceOptions) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 7: 0xe6c790 v8::StackTrace::CurrentStackTrace(v8::Isolate*, int, v8::StackTrace::StackTraceOptions) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 8: 0xc1bf36 node::GetCurrentStackTrace(v8::Isolate*, int) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 9: 0xb67b1a node::DumpJavaScriptBacktrace(_IO_FILE*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
10: 0xc1c00d node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
11: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
12: 0x7f9e97f15b56  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
13: 0x7f9e97f15c8e  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
14: 0xb21029  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
15: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
16: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
17: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
18: 0xed09c8  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
19: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
20: 0x18dcdf6  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 ELIFECYCLE  Command failed with exit code 134.

  #  Node.js[[164](https://github.com/PionixPro/summiteer-cloud/actions/runs/11857584699/job/33046303079#step:20:165)71]: void node::ResetStdio() at ../../src/node.cc:729
  #  Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1)

----- Native stack trace -----

 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 2: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 3: 0x7f097afdcb56  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 4: 0x7f097afdcc8e  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]
 5: 0xb21029  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 6: 0xb21099 node::DefaultProcessExitHandlerInternal(node::Environment*, node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 7: 0xb85a95 node::Environment::Exit(node::ExitCode) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 8: 0xed03f2 v8::internal::FunctionCallbackArguments::Call(v8::internal::CallHandlerInfo) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 9: 0xed09c8  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
10: 0xed1238 v8::internal::Builtin_HandleApiCall(int, unsigned long*, v8::internal::Isolate*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
11: 0x18dcdf6  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]

  #  Node.js[16471]: void node::ResetStdio() at ../../src/node.cc:729
  #  Assertion failed: !(err != 0) || (err == -1 && (*__errno_location ()) == 1)

----- Native stack trace -----

 1: 0xc1c005 node::Assert(node::AssertionInfo const&) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 2: 0xbc4e0e  [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 3: 0xbc4ffc node::TrapWebAssemblyOrContinue(int, siginfo_t*, void*) [/nix/store/alik5i1lm5kfm61fh3pg221czk7pmv4g-nodejs-20.16.0/bin/node]
 4: 0x7f097afda5c0  [/nix/store/wlffq5p6mxxgfap10sav3ij936jzqm59-glibc-2.39-52/lib/libc.so.6]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stalled Issues and PRs that are stalled.
Projects
None yet
Development

No branches or pull requests

5 participants