Read carefully

[kriskowal]

Fixes #1593, the bundler and compartment-mapper read many files to build a bundle and these reads are by default aggressively concurrent. This can lead to the exhaustion of file descriptors. This is generally silly because a file system does not tend to improve throughput with any amount of concurrency.

However, the compartment mapper is sufficiently general to run on any read powers, so it would be poor form to use a synchronous read function.

With this change, the compartment mapper classifies read errors in Node.js:

• those that indicate that a read was interrupted, as can occur if reading when the process receives a signal
• those that indicate that a read failed because file descriptors were exhausted
• those that indicate that a read failed because a file did not exist, or an entry existed but was not a file

…and monitors the throughput of reads. The governor implements an hybrid control loop based on AIMD (additive increase, multiplicative decrease) and CoDel (control delay).

• if throughput increases, we increase the concurrency limit by 1
• if throughput decreases, we apply a proportional decrease to the concurrency limit
• if we exhaust file descriptors, we halve the concurrency limit
• if we are interrupted, we try again and do not apply the delay against the throughput

And we also incidentally are more careful to distinguish these errors from missing entries when searching for a file (maybeRead).

A worthy critique of this PR is: perhaps instead of this level of sophistication you could just limit the Node.js read powers to a single concurrent read.

[kriskowal]

Closing in favor of #1697.