Implement the rendezvous protocol

[thomaseizinger]

This is an attempt of implementing the rendezvous protocol. This also fully includes #2081. The first commits are pretty messy but later down the track, they are quite atomic in case you prefer looking at things patch-by-patch.

A couple of design decisions that are worth mentioning:

• In-memory storage only. I see this more as a temporary thing to get an MVP out the door. Whether we want persistent storage in the form of an SQL database can be discussed in a follow-up. For example, I am not fully sold that we need it due to the ephemeral nature of registrations.
• PoW is implemented as per the drafted extension to the protocol. The extension itself is still very much WIP, so any feedback is highly appreciated.
• PoW is done on a separate thread. I did this because depending on the requested difficulty and machine power, performing the PoW can take several seconds and doing it as part of a task would essentially be blocking. I am not super happy with spawning threads inside the handler though because it makes the code non-portable to platforms like wasm which would be a shame because I think all the rest is.
• We only support one inbound and one outbound substream to a specific peer at any point.
• The test suite and substream management may appear a bit over-engineered but I wanted to experiment with some designs that might be generalizable for other protocols to make implementing protocols easier.

Things that I think we need to discuss before this can be merged:

1. How should we handle keep-alive in the handler? This is a part of rust-libp2p that I am not super familiar with so any guidance here would be much appreciated :)
2. Usage of tokio for timers. I know that rust-libp2p usually prefers async-std so we will need to come to an agreement of how this can be solved.
3. Do we want to have different modes for the behaviour such that a client could disallow registrations of other peers or should every node always also act as a rendezvous node?
4. Is the split between behaviour and handler good? Currently, the ProtocolsHandler is quite "smart" and handles all the validation (including PoW!) and substream state management. This felt right to me because it allows the behaviour to deal with decision-making only.
5. Do we want an "auto-renew" behaviour for registrations? As a behaviour that registers with a rendezvous point, we have all the necessary information to automatically renew the registration as it comes close to expiry. This might be something that all implementations want and hence should be provided together with the protocol. I am thinking of extending the register API with an argument Renew::{Never, AfterRelativeElapsed(Precent)} where AfterRelativeElapsed describes the relative time elapsed of the effective ttl after which we should renew (i.e. 50% of TTL elapsed).
6. Registering with a rendezvous node currently assumes that we want to register all of our external addresses as reported by other behaviours. This is quite opinionated but I like how it makes the interface very simple! I think we should change this so that we the user can choose between registering external addresses or providing a fixed list. Possibly in a similar way as the auto-renew functionality suggested above where the user can pass an enum Addresses::{AllExternal, Given(Vec<Multiaddress>)}

For (5) and (6), consider using a builder for a NewRegistration struct to avoid explosion of parameters.

Things we are still planning on doing (in order of importance):

• Transfer TODOs / FIXMEs from the code to this list
• Add examples for a client, a server and an application that registers itself with a rendezvous point on startup
• Fix panic on subsequent substream being injected inside inject_fully_negotiated_*
• Fix panic on of "invalid handler state" in inject_event
• Rip out PoW (see [Rendezvous] Draft proof-of-work requirement specs#334 (comment))
• Implement limit for DISCOVER requests
• Emit event for expired registrations
• Validate that incoming registration is from requesting node
• Have repeated registrations refresh existing registrations instead of duplicating them
• Replace tokio::time with wasm-timer
• Implement connection_keep_alive logic
• Handle bad cookie being sent from client
• Check the spec again and implement things like max namespace length etc (consider doing this by creating new-types for namespace and ttl to be more descriptive)
• Implement basic connection handling
  • allow kicking off a registration whilst not being connected yet, similar to libp2p-request-response
  • take Discovered responses and make the information available via addresses_of_peer to help with connection management
• Audit dependencies and remove stuff like thiserror
• Client-only mode to disallow incoming connections
  • Clarify with spec how clients should handle this (disallowing upgrades vs sending NotAvailable error?)
• Add proptests for converting between wire messages and codec messages
• Allow the user to filter the list of external addresses being registered in Rendezvous::register
• Extend spec with max message size?
• Clarify why we need the id field in Unregister message ([Rendezvous] How exactly does Unregister work? specs#335)
• Extend PeerRegistered with registration object
• Clarify with spec if there is an UnregisterResponse ([Rendezvous] How exactly does Unregister work? specs#335)

[thomaseizinger]

Can you confirm that using the same protocol string in two different handlers works if one is used for inbound and one for outbound?

I can't confirm, given that, if I am not mistaken, none of our existing protocols cover this case. E.g. #2059 uses two protocols (hop and stop), thus never running into this case.

That said, nothing comes to my mind that would prevent two handlers using the same protocol string.

Okay, I'll see if I can find the bug in my implementation then :)

[mxinden]

What do you see as critical before we can merge this?

Couple of comments below. Nothing big.

Following up on #2107 (comment), whether the swarm should hand out signed peer records, or whether each behaviour should sign their own and thus own a private key:

I would suggest we proceed here as is, with libp2p-rendezvous signing the record. Once another behaviour has the need for signed peer records, I think we should consider once more moving the logic to libp2p-swarm or libp2p-core.

[mxinden]

Ah, right, thanks. I somehow assumed other entities in a network to treat this in some other way than an opaque list of bytes.

[thomaseizinger]

@mxinden This is ready for another review!

I think I addressed all the feedback and managed to get everything working with a client/server split. Also dropped UUID to make wasm support easier.

[mxinden]

This looks good to me. Thanks for the overall work and the recent follow-ups.

Would you mind adding an entry to core/CHANGELOG.md, initialize protocols/rendezvous/CHANGELOG.md and reference it in ./CHANGELOG.md?

[thomaseizinger]

This looks good to me. Thanks for the overall work and the recent follow-ups.

Would you mind adding an entry to core/CHANGELOG.md, initialize protocols/rendezvous/CHANGELOG.md and reference it in ./CHANGELOG.md?

Done!

[mxinden]

Suggested change

	/// closed by the remote. In this case `error` is `ome`.
	/// closed by the remote. In this case `error` is `Some`.

[mxinden]

Fixed in #2218. Unfortunately I can not apply suggestions on a branch on the comit-network repository.