child_process;stdio: How does the duplex communication work?

[SEAPUNK]

Consider this code:

// parent.js

const childProcess = require('child_process')

console.log('parent:spawning')
const child = childProcess.spawn('node', ['child.js'], {
  stdio: ['ignore', 'inherit', 'inherit', 'pipe']
})

const ipc = child.stdio[3]

child.on('close', i => console.log('parent:evt:child:close', i))
child.on('error', i => console.log('parent:evt:child:error', i))
child.on('exit', i => console.log('parent:evt:child:exit', i))

ipc.on('data', i => {
  console.log('parent:evt:data', i)
  ipc.write('number 2')
})
ipc.on('close', i => console.log('parent:evt:close', i))
ipc.on('error', i => console.log('parent:evt:error', i))
ipc.on('finish', i => console.log('parent:evt:finish', i))
ipc.on('end', i => console.log('parent:evt:end', i))

console.log('parent:writing')
ipc.write('test')

setTimeout(() => process.exit(0), 10000)

// child.js

const fs = require('fs')

console.log('child:start')

setTimeout(() => {
  console.log('child:stop')
  process.exit(0)
}, 5000)

console.log('child:opening 3')

const writable = fs.createWriteStream(null, {
  fd: 3
})

writable.on('error', i => console.log('child:writable:error', i))
writable.on('close', i => console.log('child:writable:close', i))
writable.on('finish', i => console.log('child:writable:finish', i))
writable.on('drain', i => console.log('child:writable:drain', i))

const readable = fs.createReadStream(null, {
  fd: 3
})

readable.on('error', i => console.log('child:readable:error', i))
readable.on('data', i => console.log('child:readable:data', i))
readable.on('end', i => console.log('child:readable:end', i))
readable.on('close', i => console.log('child:readable:close', i))
readable.on('readable', i => console.log('child:readable:readable', i))

console.log('child:writing')
writable.write('recv')

The output is more-or-less consistently:

parent:spawning
parent:writing
child:start
child:opening 3
child:writing
parent:evt:data <Buffer 72 65 63 76>
child:readable:data <Buffer 74 65 73 74>
child:readable:data <Buffer 6e 75 6d 62 65 72 20 32>

My question is: If the communication is over one fd, which I am assuming is an anonymous pipe, how can the parent and child know to not "receive" the data they have just now written? How does that whole process work?

I've been researching file descriptors, pipes and sockets, to mostly not much avail, so I'm hoping someone here can explain it to me.

[SEAPUNK]

Quoting Wikipedia:

Full-duplex (two-way) communication normally requires two anonymous pipes.

[addaleax]

Try adding console.log('child:is-socket:fd3', fs.fstatSync(3).isSocket()) – Node is actually “cheating” here by setting up a socket, not standard unix pipes, so the file descriptors can be used as full duplexes.

[SEAPUNK]

Ah, I see -- what kind of socket is it? Does it create a file, or listen to a port?

[addaleax]

On POSIX, it’s created using socketpair():

$ strace -qfe socketpair,dup2,fork node parent.js 
parent:spawning
[pid  1254] socketpair(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0, [12, 13]) = 0
[pid  1260] dup2(16, 0)                 = 0
[pid  1260] dup2(13, 3)                 = 3
parent:writing

If you’re interested in how things work on Windows, you’ll probably have to look at libuv’s source code (at least I won’t be a great help when it comes to Windows 😄).

[bnoordhuis]

On Windows, DuplicateHandle() can magically make a HANDLE (win32 equivalent of a file descriptor) appear in another process.

The IPC channel is only used to tell the other process that it has a new HANDLE, it's not passed as auxiliary data like file descriptors are on UNIX. It's kind of neat, really.

[SEAPUNK]

So in closing, I can safely assume that the additional stdios I create when I spawn child processes will be full-duplex streams which will automatically close and clean up on either of the processes shutting down? The reason I ask is because I want to have "worker" processes, and I want to have a binary IPC between the main and worker processes.

[addaleax]

I think the answer boils down to “yes”. I am not sure how exactly that relates to how the FDs are created, but using pipe or ipc should always give you a duplex stream.

If you want to make sure that changing certain kind of behaviour is at the very least considered a semver-major change by Node, you can try to write a minimal test case and submit that as a pull request at https://github.com/nodejs/node (or look whether there’s already some test covering the behaviour).

[SEAPUNK]

Okay, thank you!