os/spawn works inconsistencly depending on what owns the pipe

[llmII]

The below works as expected, cat eventually exits (pipe ended).

(def p1 (os/spawn ["echo" "hello"] :p {:out :pipe}))
(def p2 (os/spawn ["cat"] :p {:in (p1 :out) :out :pipe}))
(:read (p2 :out) :all)
(:wait p2)

When Janet (not another process) owns the pipe however we hang!

(def p (os/spawn ["cat"] :p {:out :pipe :in :pipe}))
(:write (p :in) "hello")
(:read (p :out) :all) # hangs

I've tried quite a few ways to perhaps tell cat it's :in pipe is done, such as (close (p :in)) both before/after reading and while we close it, we still ultimately hang at the read. If we change the read to a "read amount of bytes", we can get the content. But if you are uncertain how much content is to be generated, and call read in a loop, you hang! Regardless of how much reading you manage to do or not, you can never :wait on the process, you'll hang, as the process will never end. It seems we simply don't have a way from within Janet to indicate to the child process that it won't be getting any more input and so it needs to do what it does when it doesn't have input, which is, exit, in the case of cat.

[llmII]

So, I thought to get around it by having a tertiary process called repipe that just reads from stdin and writes what it gets to stdout till it gets the string that signals EOF...

repipe.janet

(var tmp (:read stdin :line)) #originally mispasted
(def designator "END\0\0\0\0\0\n")
(while (and tmp (not= (string tmp) designator))
 (:write stdout tmp)
 (flush)
 (set tmp (:read stdin :line)))

So, lets hook it up into cat like we did with echo:

# this is echo, works
(def p1 (os/spawn ["echo" "hello"] :p {:out :pipe}))
(def p2 (os/spawn ["cat"] :p {:in (p1 :out) :out :pipe}))
(:read (p2 :out) :all)
(:wait p2)

# this is repipe
(def janet (dyn :executable))
(def designator "END\0\0\0\0\0\n")
(def repipe (os/spawn [janet "repipe.janet"] :p {:in :pipe :out :pipe}))
(def p (os/spawn ["cat"] :p {:in (repipe :out) :out :pipe :err :pipe}))

(:write (repipe :in) "hello\n")
(:write (repipe :in) designator)
(:read (p :out) :all nil 1) # returns nil, the extra args after :all are to guard against hanging
(:read (p :err) :all nil 1) # result -> @"cat: stdin: Resource temporarily unavailable\n"

cat seems to think it's stdin is already closed, which is what that error msg on stderr typically means.

[llmII]

I'll also note that if you do something like the following:

(def p2 (os/spawn ["cat"] :p {:in :pipe :out :pipe}))
(:write (p2 :in) "hello")
(:read (p2 :out) :all nil 1) #time out instead of hang

Then later call read but with a byte count instead of :all, the data was already eaten (it's not there). If you never use :all and always use a byte count, you'll get some of the data out up till you hang attempting to read from the child process that won't exit since it still thinks there's more data to be read from stdin.

[andrewchambers]

you need to close a pipe after you pass it to a child.

edit: my bad - the close was not in your example just in your comment.

[bakpakin]

Pushed a fix on master in 4a40e57 - problem was that we were leaking file descriptors.

[bakpakin]

The following program should now work:

(def p (os/spawn ["cat"] :px {:out :pipe :in :pipe}))
(:write (p :in) (string/repeat "hello" 3))
(:close (p :in))
(pp (:read (p :out) :all))

You do need to close the input, though.