Don't hit the event loop when not waiting on I/O

[brson]

I did a quick test on a message passing benchmark and avoiding the event loop was a 25% speedup. To do this right we need to keep our own I/O ref count in order to know when it's ok not to hit epoll.

[toddaaro]

Is the goal to never hit the event loop while there is no IO, or do we want to have some strategy for avoiding the event loop if there is only a little IO? The former should be really easy, the latter could be complicated.

[brson]

I have only considered the former. I don't have any ideas about the latter at the moment.

[toddaaro]

Something that might not be obvious from the short description here is that doing this means we miss all async callbacks, but that is entirely ok. Not receiving them is fine as their only function is to make sure the scheduler is active doing work, which is the case if the scheduler doesn't receive them due to being active doing work.

[alexcrichton]

Closing in favor of #8341, seems to be a bit more discussion in that bug (and it's about pretty much the same thing).