fix(net): Fix potential network hangs, and reduce code complexity

[teor2345]

Motivation

Zebra's peer handling code is complex, and it contains some potential hangs. This PR fixes some hangs, and simplifies the code.

Follow up to #7772
Close #7858

Specifications

Rust futures wake a task when any waker cloned from the most recently passed Context is woken:

When a future is not ready yet, poll returns Poll::Pending and stores a clone of the Waker copied from the current Context. This Waker is then woken once the future can make progress.
...
Note that on multiple calls to poll, only the Waker from the Context passed to the most recent call should be scheduled to receive a wakeup.

https://doc.rust-lang.org/std/future/trait.Future.html#tymethod.poll

Complex Code or Requirements

This PR contains a lot of refactors to make future changes easier.
It also eliminates some wakers entirely to simplify the code and waking logic.

Solution

Bug fixes:

• Pass context correctly to the peer set background task oneshot. (We're not waking on it now, but we might want to in future.)
• Pass context correctly to the peer client task, and the peer client request sender, so they always wake the client task. (The client task is the task that is spawned by the peer set buffer.)
• Check all ready peers for errors before sending requests to them.

Code simplification:

• Remove preselected peers, just select the peer when we have a new request. This makes waking much less complex.
• Use futures::ready! (the macro) and ? to simplify polling

Refactors:

• Refactor polling methods to return Poll, and Ok if they do something
• Make panic checking method names clearer (this can be split into its own PR)

Testing

The existing tests cover this behaviour fairly well. It is hard to check for hangs in tests.

Some tests have been updated for the new polling behaviour.

Review

This might still need some test fixes.

Reviewer Checklist

• Will the PR name make sense to users?
  • Does it need extra CHANGELOG info? (new features, breaking changes, large changes)
• Are the PR labels correct?
• Does the code do what the ticket and PR says?
  • Does it change concurrent code, unsafe code, or consensus rules?
• How do you know it works? Does it have tests?

Follow Up Work

• security: avoid hangs in the peer set and related code #7858
• Tracking: security: Limit ability of synthetic nodes to take up connection slots. Credit: Ziggurat Team  #7822

[teor2345]

This should be ready to go now, there was a minor issue with the Update future for the inventory registry, but it seemed like a good idea to make all the poll methods consistent.

[teor2345]

Does this need a full sync before it merges, or can we wait until Friday's scheduled full sync?

If we do one now, the previous time was 2d 3h 15m:
https://github.com/ZcashFoundation/zebra/actions/runs/6824538443/job/18561008809

[arya2]

Does this need a full sync before it merges, or can we wait until Friday's scheduled full sync?

I think we can wait until the next scheduled full sync.