cat: write error: Resource temporarily unavailable

[tomaszhlawiczka]

• Version: v6.11.5
• Platform: Linux blade9 4.12.12-gentoo #2 SMP Tue Nov 21 19:16:19 CET 2017 x86_64 Intel(R) Xeon(R) CPU E5450 @ 3.00GHz GenuineIntel GNU/Linux
• Subsystem: async stdio?

Problem is related to the following issue: nodejs/node-v0.x-archive#3027

Steps to reproduce with uglifyjs (2.4.10) - an error occurs randomly:

Gets some JS files:

mkdir node-test; cd node-test;
wget -q 'https://code.jquery.com/jquery-migrate-3.0.1.js'
wget -q 'https://code.jquery.com/jquery-3.3.1.min.js'

wre@blade9 ~/node-test $ ls -l
total 112
-rw-r--r-- 1 wre wre 86927 Jan 20 18:26 jquery-3.3.1.min.js
-rw-r--r-- 1 wre wre 17813 Sep 27  2017 jquery-migrate-3.0.1.js

Call uglifyjs and try to capture the output:

wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
cat: write error: Resource temporarily unavailable
wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
cat: write error: Resource temporarily unavailable
wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
wre@blade9 ~/node-test $ (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null)
cat: write error: Resource temporarily unavailable

Workaround using mktemp:

(cat jquery-migrate-3.0.1.js | (TMP="$(mktemp)"; uglifyjs 1>"$TMP" 2>/dev/null; cat "$TMP"; rm "$TMP"); cat jquery-3.3.1.min.js)|(cat - >/dev/null)

Do you know a better way to make those pipes work?

[addaleax]

@tomaszhlawiczka I think this is happening because you use the same pipe end for the uglifyjs output and the cat jquery-3.3.1.min.js output. Node.js sets its stdio pipe ends into non-blocking mode to work with async I/O, but cat expects blocking pipes.

I can’t be for sure because it doesn’t reproduce locally, but does it help if you add | cat after the 2>/dev/null bit, to pass the output of uglifyjs from a non-blocking through to a blocking pipe end?

[tomaszhlawiczka]

@addaleax thanks for reply!

(cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null | cat; cat jquery-3.3.1.min.js)|(cat - >/dev/null) - using additional | cat seems to work :-)

Thanks!

[gireeshpunathil]

link #18446

[tomaszhlawiczka]

It turns out that the issue should be addressed by bash team, right?

This bug is such an evil: who knows, writing pipe lines, which app sets FIONBIO and which does not.

[addaleax]

Hm, actually – what libuv does for TTYs is calling dup() on the fd so that it making it nonblocking doesn’t affect other programs.

Maybe we should do the same thing for pipes? /cc @nodejs/libuv

[bnoordhuis]

Node.js creates a net.Socket backed by a uv_pipe_t when the stdio fd refers to a pipe. Libuv cannot dup() the fd, that might break existing programs, but Node.js could.

[gireeshpunathil]

This issue has been resolved via #20592 and I have verified with the supplied test programs.

x=0; while [ $x -le 1000 ]; do (cat jquery-migrate-3.0.1.js | uglifyjs 2>/dev/null; cat jquery-3.3.1.min.js)|(cat - >/dev/null); x=`expr $x + 1`; echo $x; done

produced the error:
cat: write error: Resource temporarily unavailable

a hundred times in thousand iterations, but with the fix none.

Closing this, fix will be mostly available in the next current release.

[gireeshpunathil]

#20592 was reverted in master, so re-opening this.

[apapirovski]

I'm going to close this out given that it seems this might've gotten fixed again in another PR, and even if not there's not much progress here.

[gireeshpunathil]

FWIW, #24260 is the rework.