node blocks when piping to standard output. Git for Bash on Windows 10, Cygwin too.

[coderextreme]

• Version:v7.6.0, v6.10.0
• Platform: MINGW64_NT-10.0 DESKTOP-DOPK2VD 2.6.1(0.306/5/3) 2017-01-20 15:23 x86_64 Msys
  CYGWIN_NT-10.0 DESKTOP-DOPK2VD 2.6.1(0.305/5/3) 2016-12-16 11:55 x86_64 Cygwin
• Subsystem:standard input/standard output

$ cat fillbuffer.js

console.log(0);

$ yes | xargs node fillbuffer.js|tee xxx
0
0

[ I expect many more 0's going down the page and filling up the file system in the xxx file, but unfortunately, it hangs ]

Note that I may have a virus. One is quarantined. Please let me know if you can duplicate. If not, I will look elsewhere. Note that:

$ yes| xargs -L 1 echo | tee xxx
y
y
y
y
.
.
.
[ keeps going ]

works so I think the issue is with node.js

Note that

$ yes | xargs -L 1 node fillbuffer.js|tee xxx
0
0

Hangs after 2 calls to node as well. That's what I've noticed, 2 calls and then it hangs.

[coderextreme]

Note that this works, so I think the problem is with node.js (all versions, probably on Windows 10, with bash).

PROMPT$ cat fill.sh
echo 0

PROMPT$ yes | xargs sh fill.sh | tee xx
0
0
0
0
0
0
0
0
.
.
.

[seishun]

@coderextreme

It's hard to follow when the issue description is split across multiple messages. You can edit messages on GitHub, you know.

I suggest combining all your follow-up messages into one message and use code blocks to wrap your code snippets and console outputs. That will make it more likely that someone will look into your issue.

[coderextreme]

@seishun:

Here is the most minimal thing I've found so far which hangs. This needs to be fixed, ASAP! Cmd prompt works fine. Which indicates there's some issue between bash and node that people haven't figured out yet.

PROMPT$ cat fillbuffer.js
console.log(0);

PROMPT$ yes | xargs node fillbuffer.js|tee xxx
0
0
[hung]

If you cannot understand this code, I don't think I want you to fix it. If you cannot duplicate the hang in bash (any version) let me know the version of bash you are using. Thanks!

[addaleax]

@coderextreme Friendly reminder that:

• There is a code of conduct, and if your comments are going to be bordering on personal attacks that’s just going to get you blocked (“We will exclude you from interaction if you insult, demean or harass anyone”). I’ve edited your last comment accordingly.
• @seishun’s requests are perfectly valid. Remember that, unless you’re going to fix this bug yourself, you are asking somebody else to do that work for you for free. Wrapping code in code blocks does increase readability massively.

[coderextreme]

@addaleax the fix beyond my level and requires knowledge of bash, windows and node. If someone cannot understand a bash prompt, they don't know enough about bash to solve the problem. I won't fix the bug, I will either walk away from node, windows or bash in that order. There are other perfectly good languages and environments I can work in without getting a headache. This problem caused me to upgrade my bash terminal, and now I can't copy out of vim. I am PISSED. This is a SERIOUS problem that I already worked a long time on and gave you a perfectly simple test case for, after whittling for hours. There was enough problems getting bash to output to a pipe apparently on windows.

[rvagg]

This isn't a forum for venting, either be constructive or move along. As we don't have many full-time Windows users among us you're going to have to be patient for someone to be able to even try to replicate this. Expecting quick action, being abusive and taking out your frustration on this repository are not going to speed up resolution, more likely it'll have the opposite effect. If you'd like to be patient and constructive then we'd be happy to work with you to try and figure out if this is in fact a Node problem or something else.

[coderextreme]

I am searching for windows users in the bash forum on freenode.
I cannot get into the nodejs forum.

[mcollina]

@coderextreme can you please clarify what should be the expected vs actual behaviour and edit the top description? This issue is hard to follow.

[coderextreme]

Do you recommend using nashorn on windows, or will it have the same problem? I might try to duplicate it.

[coderextreme]

maybe not nashorn...no console. sigh!

[gareth-ellis]

I can recreate it on windows running Cmder / GNU bash, version 4.3.46(2)-release.

The key is the piping output,

yes|xargs node fillbuffer.js runs all day long, as soon as you pipe the output somewhere, that's when the problems start.

[coderextreme]

Nashorn can do it! Go node team! Make us all proud!

$ yes | xargs jjs fillbuffer.js -- | tee xxx
0
0
0
0
0
0
0
0
.
.
.

$ cat fillbuffer.js
print(0);

[coderextreme]

@gareth-ellis you may be right, but look at this:

$ cat fillbuffer.js
for (var i = 0; i < 1000000; i++) {
        console.log(0);
}

$  node fillbuffer.js|tee xxx
stdout is not a tty

This could be true source of the issue. I think bash had the same issue

[gibfahn]

@coderextreme so you are using cygwin? Do you see the same issue using CMD (or Git Bash)?

PS I edited your posts so I could read them.

[coderextreme]

@gibfahn I've only run it once in CMD and it works fine. It requires changing my PATH every time I start CMD, so I don't bother much. Did you want me to try something?

[gibfahn]

I'm not an expert on Windows, but I know that there are problems with node not running as a tty on Cygwin, there have been quite a few issues raised about that. The easy way to check is to install and try Git Bash, which does something else with ttys (I think it aliases node to winpty node).

See: #3006

You can also try the winpty option mentioned in that issue.

Note that cygwin isn't a supported platform

[seishun]

$  node fillbuffer.js|tee xxx
stdout is not a tty

node.exe works, so it's an unrelated problem caused by winpty. See here for an explanation.

[coderextreme]

The problem first showed up in git bash (or git for windows) It's the MINGW64 version. I upgrade and that didn't solve the problem. I'm still running git bash (has the 4 colored windows-like icon in the corner).

[coderextreme]

@seishun; This still fails:

$ yes | xargs node.exe fillbuffer.js | tee xxx
0
0
[hang]

$ cat fillbuffer.js
console.log(0);

[coderextreme]

And:

$ yes | xargs winpty node.exe fillbuffer.js | tee xxx
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty
stdin is not a tty

[coderextreme]

So what you're telling me is, if I want to use pipes, I should use jjs (nashorn) until they shut that down too?

[coderextreme]

I use both Cygwin and MingW64 (Git Bash). The dual pipe hangs in both.

[seishun]

This also reproduces the issue, without xargs or tee:

$ (node.exe fillbuffer.js && node.exe fillbuffer.js) | node.exe read.js
<Buffer 30 0a>
<Buffer 30 0a>

(hangs here)

read.js:

process.stdin.on('data', console.log);

[Trott]

I could be wrong, but offhand, these lines of code seem relevant:

node/lib/net.js

Lines 149 to 153 in 8e8fbd8

	if ((options.fd == 1 || options.fd == 2) &&
	(this._handle instanceof Pipe) &&
	process.platform === 'win32') {
	// Make stdout and stderr blocking on Windows
	var err = this._handle.setBlocking(true);

[Trott]

/cc @orangemocha based on 20176a9 perhaps?

[Trott]

Also seemingly relevant: nodejs/node-v0.x-archive#3584

[coderextreme]

I did test on 0.12.18. The problem was there.

[coderextreme]

@seishun do you need a:

process.stdin.on('end', ...

In read.js
and maybe then it will exit?

I verified that the following read.js did not improve things:

process.stdin.on('data', console.log);
process.stdin.on('end', console.log);

John

[Fishrock123]

node blocks when piping to standard output. ... Windows ...

Yes. See this Documentation note for all the details: (Not sure that this has made it into a release yet, so maybe not yet on nodejs.org in this form.) https://github.com/nodejs/node/blob/master/doc/api/process.md#a-note-on-process-io

See #10884 (comment) for even more back info.

If you would like to manually change that behavior (pipes on Windows), use this. I make no guarantee things will not break.

if (process.stdout._handle) process.stdout._handle.setBlocking(false);

Given this is already known and documented, is there any reason to keep this particular issue open?

[seishun]

Per libuv/libuv#238, the WriteFile call here should post a completion packet which is then retrieved here. But evidently it doesn't get posted when running node.exe the second time using xargs or command grouping in Git Bash, which causes the pGetQueuedCompletionStatusEx call to never return.

Since this doesn't happen in cmd, I'm inclined to believe it's a Git Bash bug, e.g. it doesn't create a pipe properly. Thus I propose closing this issue, unless someone is willing to reproduce the same problem using standard Windows tools and/or C++/Node.js code.

cc @bnoordhuis @orangemocha @piscisaureus

[Fishrock123]

Soooo... use Hyper.is on windows I guess?

[coderextreme]

The solution was to put:

process.exit();

At the end of the file.

Thanks for all your help! If I find another issue with a more complex case, I will post it.