Mempool component and storage

[dconnolly]

Motivation

First iteration of the main Mempool as a service that wraps a mempool::Storage manager. The crawler and tx download and verify stream will be internal to this component (privileged) while consumers of mempool will query it via the Request/Response protocol.

Specifications / Designs

https://docs.google.com/document/d/10PP6wKlnS5gannRph_EQ8V9diR7byddf3W2YFZ564FU/edit#heading=h.1v3bcppsa6wq
https://jamboard.google.com/d/1HUhdqx8afJUqgiBdBcjCw0aAfDod9R9KBJeAH24U1Qg/viewer?f=0

Solution

Resolves #2589, #2591, #2620

Review

Reviewer Checklist

• Code implements Specs and Designs
• Tests for Expected Behaviour
• Tests for Errors

Follow Up Work

[dconnolly]

I think this also resolves #2620 and #2608 and #2607

[oxarbitrage]

I added a very basic test for the storage in 4ea0067 and i fixed a bug in 08269b4 (The drain should be called only after the mempool is full or it will panic).

Another issue i found that maybe was already discussed is that all transactions get moved from the verified into rejected as new transactions get in. This means we can end with a huge amount of them in this state. Is there a purpose for keeping all the ids here ?

[teor2345]

Just some typos, clarifications, and name suggestions

[teor2345]

Another issue i found that maybe was already discussed is that all transactions get moved from the verified into rejected as new transactions get in. This means we can end with a huge amount of them in this state. Is there a purpose for keeping all the ids here ?

We keep IDs because we don't want to re-download, re-verify, or re-store transactions we've recently rejected.

I've added a task to put a size limit on the rejected list, but I don't think it should block this PR.
(I'd really like to get this PR merged, so we can start connecting it up to other modules.)

Also we'll clear the transactions and rejected IDs every time we reset the mempool, for example, when we switch chain forks. (Approximately every 300 blocks.)

[teor2345]

I'd like to fix some indexing bugs in the tests before we merge.

[oxarbitrage]

All comments were addressed, i think we can do more work in here but is good enough IMHO to get started so i will merge it as it is also blocking other PRs.