readbytes stops prematurely

[denizyuret]

I cannot get the following command the complete:

a=readbytes(`wget -O- http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz` |> `zcat`)

It works without zcat. It works with smaller files (the file causing the problem above is ~9MB). It works in regular shell outside of Julia. I replicated the behavior on v0.3, v0.4 on linux and osx.

[denizyuret]

Hm, it doesn't seem to have anything to do with the download or the pipe, I can replicate the same behavior with a local file:

a=readbytes(`zcat train-images-idx3-ubyte.gz`)

Smaller files work, so this is probably hitting an OS limit?

[ihnorton]

I can reproduce on linux, but I believe this is a dupe of #8789.

[stevengj]

i.e. the problem is an OS limitation on writing and reading in the same kernel thread as in #8789? However, in this case the writing is being done by a separate process; can't that be run asynchronously in a different kernel thread?

[jakebolewski]

closing as a dupe of #8789

[nbaum]

I think this is actually a dupe of #10655 :)