Writing more than 64 kb to stdout

[philip4096]

I am calling something that writes more than 64 kb to stdout from IJulia Notebook and the process stalls. Less than 64 kb works and doing the same thing on a worker process (or outside Ijulia notebook) works fine.

[stevengj]

I can't reproduce:

randstring(n) = ASCIIString([uint8(rand('#':'z')) for i = 1:n])
print(randstring(2^17))

works fine for me (128kiB output). Does this example fail for you? What is the output of versioninfo()?

[philip4096]

Your example works fine. versioninfo() below. I am using PyCall and the win32com.client python module to call COM-functions. Since exactly the same thing works in a standalone julia process I didn't think it was a python issue.

Julia Version 0.3.1
Commit c03f413 (2014-09-21 21:30 UTC)
Platform Info:
System: Windows (i686-w64-mingw32)
CPU: Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz
WORD_SIZE: 32
BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Nehalem)
LAPACK: libopenblas
LIBM: libopenlibm
LLVM: libLLVM-3.3

[stevengj]

What about:

ccall(:puts, Cint, (Ptr{Uint8},), randstring(2^17))

? That works for me, but maybe there is a problem with the libc I/O interactions on your machine?

As shown from my example code, there is no inherent problem in IJulia and IPython with more than 64kB standard output, so I'm not sure where the problem lies on your machine.

[stevengj]

Actually, now I see a problem on my machine:

ccall(:puts, Cint, (Ptr{Uint8},), string([1:2^16]))

returns -1 (EOF) and the output is clearly truncated, so something is going wrong with libc I/O for me too (MacOS 10.9). (Whereas it works in the terminal, so it has something to do with the I/O redirection in IJulia.)

cc: @vtjnash

[stevengj]

I'm closing this in favor of JuliaLang/julia#8789, since I was able to reproduce the problem outside IJulia by redirecting stdout.

[philip4096]

OK, good, thank you!

[stevengj]

Re-opening this issue as it is not a Julia problem per se, it is an OS limitation for reading and writing a buffer device from the same thread.

The only solution would be to somehow instantiate a genuine OS thread that is responsible for reading the IO buffers; this is theoretically possible but will require some work (possibly at the C level).