EPIPE with the repl and cat? #3027

TooTallNate · 2012-03-28T17:48:45Z

Basically, I'm seeing EPIPE errors when a "large" amount of data gets printed to the output stream when using the repl over pipes. To reproduce, inspect a largish object in the repl, like process or even process.env. This is on OS X, happens on both Terminal and iTerm2.

☮ ~/node (master) ⚡ ./node | cat
> process
{ EventEmitter: [Function: EventEmitter],
  title: './node',
  version: 'v0.7.7-pre',
  assert: [Function],
  config: 
   { target_defaults: 
      { cflags: [],
        default_configuration: 'Release',
        defines: [],
        include_dirs: [],
        libraries: [Object] },
     variables: 
      { host_arch: 'x64',
        node_install_npm: true,
        node_install_waf: true,
        node_prefix: '',
        node_shared_v8: false,
        node_shared_zlib: true,
        node_use_dtrace: false,
        node_use_openssl: true,
        node_use_system_openssl: true,
        strict_aliasing: true,
        target_arch: 'x64',
        v8_use_snapshot: true } },
  moduleLoadList: 
   [ 'Binding evals',
     'Binding natives',
     'NativeModule events',
     'NativeModule buffer',
     'Binding buffer',
     'NativeModule assert',
     'NativeModule util',
     'NativeModule path',
     'NativeModule module',
     'NativeModule tty',
     'NativeModule net',
     'Nativewrite error


events.js:48
        throw arguments[1]; // Unhandled 'error' event
                       ^
Error: write EPIPE
    at errnoException (net.js:685:11)
    at Object.afterWrite [as oncomplete] (net.js:510:18)

So that's bad. I've also seen cat print to the screen: cat: write error: Resource temporarily unavailable. Interestingly, if we turn stdin into a pipe as well, then it seems to work over cat (the -i flag on node makes it enter the repl, rather than interpret stdin as a piped-in JS file):

echo process | ./node -i | cat

So this is a little strange to me. Naively, it seems like something possibly on our (libuv?) end, but I really don't know the fine details here.

Another interesting tidbit is that I get essentially the same behavior when running a repl over an HTTP server, connecting using curl. It works seemingly ok, but if you print out process.env for example, then curl crashes with: curl: (23) Failed writing body (0 != 2020). See this gist: https://gist.github.com/2053342

The text was updated successfully, but these errors were encountered:

TooTallNate · 2012-03-29T01:52:37Z

This also dates back to at least node v0.4.12, so we know it's not necessarily related to libuv.

Southern · 2012-03-31T14:18:27Z

Seems to work fine for me.

~$ node | cat
> process
{ title: 'node',
  EventEmitter: [Function: EventEmitter],
  version: 'v0.7.7-pre',
  assert: [Function],
  moduleLoadList: 
   [ 'Binding evals',
     'Binding natives',
     'NativeModule events',
     'NativeModule buffer',
     'Binding buffer',
     'NativeModule assert',
     'NativeModule util',
     'NativeModule module',
     'NativeModule path',
     'NativeModule tty',
     'NativeModule net',
     'NativeModule stream',
     'NativeModule timers',
     'Binding timer_wrap',
     'NativeModule _linklist',
     'Binding tty_wrap',
     'NativeModule vm',
     'NativeModule fs',
     'Binding fs',
     'Binding constants',
     'NativeModule readline',
     'Binding pipe_wrap' ],
  config: 
   { target_defaults: 
      { cflags: [],
        default_configuration: 'Debug',
        defines: [],
        include_dirs: [],
        libraries: [] },
     variables: 
      { host_arch: 'x64',
        node_install_npm: true,
        node_install_waf: true,
        node_prefix: '',
        node_shared_v8: false,
        node_shared_zlib: false,
        node_use_dtrace: false,
        node_use_openssl: true,
        node_use_system_openssl: false,
        strict_aliasing: true,
        target_arch: 'x64',
        v8_use_snapshot: true } },
  _tickCallback: [Function],
  versions: 
   { http_parser: '1.0',
     node: '0.7.7-pre',
     v8: '3.9.17',
     ares: '1.7.5-DEV',
     uv: '0.6',
     zlib: '1.2.3',
     openssl: '0.9.8r' },
  arch: 'x64',
  nextTick: [Function],
  platform: 'linux',
  stdout: [Getter],
  argv: [ 'node' ],
  stderr: [Getter],
  execArgv: [],
  stdin: [Getter],
  openStdin: [Function],
  env: 
   { [...snip...] },
  pid: 6329,
  exit: [Function],
  features: 
   { debug: false,
     uv: true,
     ipv6: true,
     tls_npn: true,
     tls_sni: true,
     tls: true },
  kill: [Function],
  execPath: '/usr/local/bin/node',
  addListener: [Function],
  on: [Function],
  debugPort: 5858,
  removeListener: [Function],
  _needTickCallback: [Function],
  reallyExit: [Function],
  abort: [Function],
  chdir: [Function],
  cwd: [Function],
  umask: [Function],
  getuid: [Function],
  setuid: [Function],
  setgid: [Function],
  getgid: [Function],
  _kill: [Function],
  _debugProcess: [Function],
  _debugPause: [Function],
  _debugEnd: [Function],
  hrtime: [Function],
  dlopen: [Function],
  uptime: [Function],
  memoryUsage: [Function],
  uvCounters: [Function],
  binding: [Function] }

Just pulled from master again, to make sure it wasn't a recent bug.

~$ node | cat
> process
{ title: 'node',
  EventEmitter: [Function: EventEmitter],
  assert: [Function],
  version: 'v0.7.8-pre',
  config: 
   { target_defaults: 
      { cflags: [],
        default_configuration: 'Debug',
        defines: [],
        include_dirs: [],
        libraries: [] },
     variables: 
      { host_arch: 'x64',
        node_install_npm: true,
        node_install_waf: true,
        node_prefix: '',
        node_shared_v8: false,
        node_shared_zlib: false,
        node_use_dtrace: false,
        node_use_openssl: true,
        node_use_system_openssl: false,
        strict_aliasing: true,
        target_arch: 'x64',
        v8_use_snapshot: true } },
  moduleLoadList: 
   [ 'Binding evals',
     'Binding natives',
     'NativeModule events',
     'NativeModule buffer',
     'Binding buffer',
     'NativeModule assert',
     'NativeModule util',
     'NativeModule module',
     'NativeModule path',
     'NativeModule tty',
     'NativeModule net',
     'NativeModule stream',
     'NativeModule timers',
     'Binding timer_wrap',
     'NativeModule _linklist',
     'Binding tty_wrap',
     'NativeModule vm',
     'NativeModule fs',
     'Binding fs',
     'Binding constants',
     'NativeModule readline',
     'Binding pipe_wrap' ],
  versions: 
   { http_parser: '1.0',
     node: '0.7.8-pre',
     v8: '3.9.24.7',
     ares: '1.7.5-DEV',
     uv: '0.6',
     zlib: '1.2.3',
     openssl: '0.9.8r' },
  _tickCallback: [Function],
  nextTick: [Function],
  arch: 'x64',
  stdout: [Getter],
  platform: 'linux',
  stderr: [Getter],
  argv: [ 'node' ],
  stdin: [Getter],
  execArgv: [],
  openStdin: [Function],
  env: 
   { [..snip..] },
  exit: [Function],
  pid: 14164,
  features: 
   { debug: false,
     uv: true,
     ipv6: true,
     tls_npn: true,
     tls_sni: true,
     tls: true },
  kill: [Function],
  execPath: '/usr/local/bin/node',
  addListener: [Function],
  on: [Function],
  debugPort: 5858,
  _needTickCallback: [Function],
  removeListener: [Function],
  reallyExit: [Function],
  abort: [Function],
  chdir: [Function],
  cwd: [Function],
  umask: [Function],
  getuid: [Function],
  setuid: [Function],
  setgid: [Function],
  getgid: [Function],
  _kill: [Function],
  _debugProcess: [Function],
  _debugPause: [Function],
  _debugEnd: [Function],
  hrtime: [Function],
  dlopen: [Function],
  uptime: [Function],
  memoryUsage: [Function],
  uvCounters: [Function],
  binding: [Function] }

Have you tried seeing if the SIGPIPE event is triggered? process.on('SIGPIPE', function() { throw new Error('Pipe closed.'); });

mmalecki · 2012-03-31T15:31:37Z

Can reproduce on Mac OS X. Works without problems on Linux.

TooTallNate · 2012-03-31T18:31:38Z

@Southern There does not appear to be a "SIGPIPE" event that happens.

TooTallNate · 2012-03-31T18:37:15Z

Ok this does indeed seem to be an OS X problem. I tried it on Solaris, Windows, and Linux and it worked as expected in all those places. I didn't try my REPL over curl example on all those platforms, but I think that is a different problem now.

@mmalecki @Southern Can you guys try out the REPL over curl example as well? Requires node v0.7.7.

Southern · 2012-03-31T19:32:17Z

Not sure that my testing it would be of any help, since I am using Linux.

TooTallNate · 2012-03-31T19:46:49Z

@Southern I have a feeling that the "repl-over-curl" situation is a little different. I was able to reproduce a similar crash in curl by printing out a large object like global to the repl, on both OS X and Ubuntu.

Interestingly Windows was ok, though the color codes do not get interpreted properly since curl doesn't know how to handle them.

mhart · 2012-05-03T08:53:11Z

Is #3211 a simple test case of the same thing? (edit: perhaps not - SIGPIPE is being triggered in my case) (edit again: well... sometimes it is, sometimes it's not - could be the process is terminating before it's caught in some cases)

vvo · 2012-10-17T16:13:53Z

Have reproduced this on Ubuntu 12.04 too when too much content get written to repls and lot of connections.

https://gist.github.com/3906440: test files that almost always give

Error: write EPIPE
    at errnoException (net.js:769:11)
    at Object.afterWrite (net.js:593:19)

Node 0.8.12

-> % uname -ra
Linux [...] 3.2.0-32-generic [...] Ubuntu [...] x86_64 x86_64 x86_64 GNU/Linux

Maybe this is just an inode problem? dunno

bnoordhuis · 2012-10-23T23:55:49Z

I'm going to close this because it's not really a node bug. When you run node | cat the following happens:

node reads from the tty and writes to the pipe.
cat reads from the pipe and writes to the tty.
node puts the tty in non-blocking mode.
When cat tries to write, it eventually gets an EAGAIN (meaning: tty output buffer full). It's not equipped to handle that and exits.
The read end of the pipe is closed. When node next tries to write, it gets an EPIPE error.

tl;dr It's a bug in cat, it should switch to polling mode when it sees EAGAIN but it doesn't.

It may be possible to work around that by reopening /dev/tty in node but I'm a) not sure that'll work at all, and b) that it'll work on all supported Unices.

TooTallNate · 2012-12-04T23:00:31Z

@bnoordhuis I appreciate the explanation and it makes perfect sense to me. I'm confused, though, at why only OS X seems to exhibit this behavior, and the other OSs seem alright.

bnoordhuis · 2012-12-05T02:34:51Z

OS X seems to exhibit this behavior, and the other OSs seem alright.

It happens on Linux as well but the tty output buffer is a lot bigger there, on the order of 256 kB.

On OS X, it's around 4 kB (or 8 or 16 kB - a lot smaller anyway) so it fills up faster.

TooTallNate · 2012-12-05T02:36:42Z

I learn so much from you Ben :)

On Tue, Dec 4, 2012 at 6:35 PM, Ben Noordhuis notifications@github.comwrote:

OS X seems to exhibit this behavior, and the other OSs seem alright.

It happens on Linux as well but the tty output buffer is a lot bigger
there, on the order of 256 kB.

On OS X, it's around 4 kB (or 8 or 16 kB - a lot smaller anyway) so it
fills up faster.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/3027#issuecomment-11025814.

bnoordhuis closed this as completed Oct 23, 2012

bnoordhuis mentioned this issue Oct 24, 2012

unix: re-open tty stdio fds as /dev/tty? joyent/libuv#601

Closed

tomaszhlawiczka mentioned this issue Apr 5, 2018

cat: write error: Resource temporarily unavailable nodejs/node#19838

Closed

Random-Liu mentioned this issue Oct 23, 2019

Test got terminated randomly after upgrading to gcr.io/k8s-testimages/kubekins-e2e:v20191017-ac4b4b5-master kubernetes/test-infra#14938

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

EPIPE with the repl and cat? #3027

EPIPE with the repl and cat? #3027

TooTallNate commented Mar 28, 2012

TooTallNate commented Mar 29, 2012

Southern commented Mar 31, 2012

mmalecki commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

Southern commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

mhart commented May 3, 2012

vvo commented Oct 17, 2012

bnoordhuis commented Oct 23, 2012

TooTallNate commented Dec 4, 2012

bnoordhuis commented Dec 5, 2012

TooTallNate commented Dec 5, 2012

EPIPE with the repl and cat? #3027

EPIPE with the repl and cat? #3027

Comments

TooTallNate commented Mar 28, 2012

TooTallNate commented Mar 29, 2012

Southern commented Mar 31, 2012

mmalecki commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

Southern commented Mar 31, 2012

TooTallNate commented Mar 31, 2012

mhart commented May 3, 2012

vvo commented Oct 17, 2012

bnoordhuis commented Oct 23, 2012

TooTallNate commented Dec 4, 2012

bnoordhuis commented Dec 5, 2012

TooTallNate commented Dec 5, 2012