Avoid starvation from FuturesUnordered::poll_next

[jonhoo]

This fixes #2047.

[cramertj]

I'm unsure about this. Overall, I think we need some better way to address this issue globally, as this behavior is also an issue with every other kind of Future/Stream/asynchronous function. If a task has work remaining that can be done, it will never yield to the executor. Making every single looping combinator track a maximum poll count seems like a possibility, but it seems like the kind of practice we'd want some discipline around, rather than picking arbitrary integers which will stack exponentially (with this PR, the limit of leaf-future polls for a FuturesUnordered containing futures which poll a FuturesUnordered is now 32^2=1024, and another layer would make it 32768). Perhaps @carllerche has thoughts?

[jonhoo]

Yup, I agree it's not ideal, which is why I also proposed tokio-rs/tokio#2160 as a more "complete" fix. I actually argued the exact same point about exponentials in the tokio Discord a few hours ago as a reason why we ultimately want a more global mechanism :p I do think we need a solution in the short run too though, as global preemption is probably something that will take longer to settle, and this is certainly an issue right now (for example, I am running into it in Noria as we speak).

[carllerche]

I agree w/ @cramertj. IMO this should be tracked at the runtime task level and leaf resource operations should be the ones to impact the poll budget.

[cramertj]

Merging-- see discussion ongoing in #2047

[jonhoo]

@cramertj What's the procedure for doing a patch release with this change?

[cramertj]

@jonhoo I'll put one out this week! Are you under time pressure? If so, I'll can try and do one tomorrow (the 28th).

[jonhoo]

Nope, no rush on my end. I'm using it in https://github.com/mit-pdos/noria, but we're fine using patches in Cargo.toml to git for the time being :)

[jonhoo]

@cramertj Any update on a release for this? Would be nice to get rid of the patches.

[cramertj]

Yup, I'm doing one today!

[jsancio]

Thanks for the feature @jonhoo. How are executor expected to use this feature? When checking for events (e.g. epoll) executors need to know if they should:

1. Block forever until an event triggers because there aren't any task in the ready queue.
2. Check for events without blocking because there are tasks in the ready queue.

FuturesUnordered doesn't expose this information. In other words, now FutureUnordered::poll_next can return Poll::Pending even if there are tasks in the ready queue.

[jonhoo]

@jsancio I'm not entirely sure I follow? Whenever FuturesUnordered does this "forced yield", it also calls .wake() on its waker to ensure that it gets poll again. As long as the executor respects calls to wake during poll, it should be good. It shouldn't go to sleep when waiting for events, since it has at least one ready task (the FuturesUnordered). Does that answer your question?

[jsancio]

Thanks @jonhoo. Yeah, it makes sense. I missed the part that wake() was getting called even it if it was returning Poll::Pending.