Performance regression in pipe after v0.11.3 on OS X

[mhart]

(extracted from #3429 because that's specifically about the disparity between stdin and spawn)

This graph shows the performance of cat file | node readStdin.js on different node.js versions on OS X 10.11:

This was generated from 11 runs of the following program after cat/piping a 1GB file generated with head -c 1000000000 /dev/zero > file:

stdinPipe(function(err, time) {
  if (err) return console.error(err.stack || err)
  console.log([process.version, time].join(','))
})

function stdinPipe(cb) {
  var bytesRead = 0
  var time = process.hrtime()

  process.stdin.resume()
  process.stdin.on('data', function(data) {
    bytesRead += data.length
  })
  process.stdin.on('error', cb)
  process.stdin.on('end', function() {
    if (bytesRead != 1000000000) return cb(new Error('Incorrect bytes read: ' + bytesRead))
    var diff = process.hrtime(time)
    cb(null, 1000 / (diff[0] + diff[1] / 1e9))
  })
}

Now interestingly, as far as I can tell this seems to be a specifically pipe-related problem, because if the same program is run with node readStdin.js < file, no such difference exists:

[silverwind]

v0.11.3-release...v0.11.4-release

[jasnell]

is this still an issue in v4, v5 or master?

[mhart]

You can see v4.2.1 there in the graph – would there have been anything that would've fixed it?

Can you reproduce?

[MylesBorins]

I'm going to investigate this

[Fishrock123]

Sounds like a duplicate of #3477 (comment) -- @addaleax would you mind checking?

[mhart]

@Fishrock123 just to be clear – these two issues were spun out because this one is Mac OS X specific.

[addaleax]

Yeah, the symptoms here don’t match that either. This one also looks more fixable, I guess?

[Trott]

I don't have any reason to believe this was fixed in Node.js 6.x or 8.x, but does anyone want to run a comparison along with the versions already in the graph above so we know for sure?

[mhart]

This is definitely still an issue btw, the needle's hardly moved since v0.12. The performance of v8.4.0 is only 35% of what it is in v0.10.48.

See #3429 for how this chart was generated

[mhart]

Actually, scratch that – pretty sure this is another case of #1671

If I add gc(true) calls every 100 data events, this is how the chart looks:

So the gap between spawn and stdin is still there (which is what #3429 is about) – but there's no drop in performance between versions if GC is fixed (as I guess is proposed)

Closing as a duplicate of #1671