Feature request: Horizontal peer-to-peer event-loop messaging w/ process.send (workers)

[NawarA]

To make Node.js concurrency model as efficient as possible, we need to be able to use process.send to send messages to an independent (not master) Node.js event loop.

Similar to the Open Web Standard of postMessage, a frame can postMessage to any frame, given proper reference. We need the approach for our "frames", which in Node.js, are forked event loops. As a developer, I should be able to have forked Child and Worker event loops send(message) to one another, without having middle-manning between whoever created the child.

For example, if Master event loop has a forked process, then one of the Master's worker event-loops, should be able to directly communicate to the Forked process, without having to first process.send(message) to Master.

The benefit of this is that it allows Node.js logic to more easily process things in parallel, in a truly async model. We all know that nextTick and timers simply push an item down in the queue to be later processed on the same event-loop, which obviously is not truly "async" in the generic definition...its more appropriately doThisLater logic, than processThisInTheBackground logic and report to me when done.

The final benefit I can think of is that it allows to share memory spaces. Java is able to share memory spaces efficiently across thousands of threads, which one reason why Java is highly adopted and efficient at high-concurrency. Meanwhile, we, in the Node.js community, cannot access shared spaces thanks to Fork... and we could if our event loops could communicate horizontally, instead of today, where send(message) only works on the Master that created them.

We should change the API from this:
process.send(message[, sendHandle[, options]][, callback])
to:
process.send(message[, fileDescriptor[, sendHandle][, options]][, callback])

where fileDescriptor referrers to a Node.js event loop's system identifier.

I know the IPC approach that we have may make this feature request interesting to implement, but I imagine there is way. Improving Node.js's ability to process things in the background would greatly benefit high-concurrency services that process thousands of requests a second, such as my own, and many other high-concurrency services in our community.

[Fishrock123]

Sounds like Workers. I don't guarantee it will be exactly the same API.

I think @bnoordhuis is working on this. See also #2133

[Trott]

#2133 was closed without landing, although it seems that someone could come along and pick it up.

/cc @bnoordhuis @cjihrig @mcollina for comment

[mcollina]

@Trott there is active discussion in nodejs/worker#1, cc @addaleax.
I think this issue can be closed, as there are more up-to-date ones with lively discussions on how to get there.