Waku v2 Track 1 - Move to libp2p epic

[oskarth]

Part of #39

Moving to PubSub over libp2p wouldn't improve amplification per se, but it would be stepping stone. Why? It paves the way for GossipSub, and would be a checkpoint on this journey. Additionally, FloodSub and GossipSub are compatible, and very likely other future forms of PubSub such as GossipSub 1.1 (hardened/more secure), EpiSub, forwarding Kademlia / PubSub over Kademlia, etc. Not to mention security This would also give us access to the larger libp2p ecosystem (multiple protocols, better encryption, quic, running in the browser, security audits, etc, etc), as well as be a joint piece of infrastructured used for Eth2 in Nimbus. More wood behind fewer arrows.

See more on libp2p PubSub here: https://docs.libp2p.io/concepts/publish-subscribe/

As part of this move, there are a few individual pieces that are needed.

1. FloodSub

This is essentially what Waku over libp2p would look like in its most basic form.

One difference that is worth noting is that the app topics would not be the same as Waku topics. Why? In Waku we currently don't use topics for routing between full nodes, but only for edge/light nodes in the form of topic interest. In FloodSub, these topics are used for routing.

Why can't we use Waku topics for routing directly? PubSub over libp2p isn't built for rare and ephemeral topics, and nodes have to explicitly subscribe to a topic. See topic sharding section for more on this.

Moving to FloodSub over libp2p would also be an opportunity to clean up and simplify some components that are no longer needed in the Waku v1 protocol, see point below.

Very experimental and incomplete libp2p support can be found in the nim-waku repo under v2: https://github.com/status-im/nim-waku

2. Simplify the protocol

Due to Waku's origins in Whisper, devp2p and as a standalone protocol, there are a lot of stuff that has accumulated (https://specs.vac.dev/specs/waku/waku.html). Not all of it serves it purpose anymore. For example, do we still need RLP here when we have Protobuf messages? What about extremely low PoW when we have peer scoring? What about key management / encryption when have encryption at libp2p and Status protocol level?

Not everything has to be done in one go, but being minimalist at this stage will the protocol lean and make us more adaptable.

The essential characteristic that has to be maintained is that we don't need to change the upper layers, i.e. we still deal with (Waku) topics and some envelope like data unit.

3. Core integration

As early as possible we want to integrate with Core via Stimbus in order to mitigate risk and catch integration issues early in the process. What this looks like in practice is some set of APIs, similar to how Whisper and Waku were working in parallel, and experimental feature behind a toggle in core/desktop.

4. Topic interest behavior

While we target full node traffic here, we want to make sure we maintain the existing bandwidth requirements for light nodes that Waku v1 addressed (https://vac.dev/fixing-whisper-with-waku). This means implementing topic-interest in the form of Waku topics. Note that this would be separate from app topics notes above.

5. Historical message caching

Basically what mailservers are currently doing. This likely looks slightly different in a libp2p world. This is another opportunity to simplify things with a basic REQ-RESP architecture, as opposed to the roundabout way things are now. Again, not everything has to be done in one go but there's no reason to reimplement a poor API if we don't have to.

Also see section below on adaptive nodes and capabilities.

6. Waku v1 <> Libp2p bridge

To make the transition complete, there has to a be bridge mode between current Waku and libp2p. This is similar to what was done for Whisper and Waku, and allows any nodes in the network to upgrade to Waku v2 at their leisure. For example, this would likely look different for Core, Desktop, Research and developers.

[oskarth]

Current state in nim-waku for v2 / FloodSub is a very basic PoC over libp2p with most Waku specific functionality missing:

Node/simulation: https://github.com/status-im/nim-waku/tree/master/waku/node/v2
Protocol: https://github.com/status-im/nim-waku/tree/master/waku/protocol/v2

[oskarth]

Started initial raw spec here to have something more concrete to talk about vacp2p/rfc#144

[oskarth]

Some links for easy reference

• Planned Stimbus shim nim-status-go shim status-im/nim-status#5
• Waku v1 RPC API (implemented in nim-waku, subset of status-go) https://github.com/vacp2p/specs/blob/master/specs/waku/waku-rpc-api.md
• status-react JSON RPC https://github.com/status-im/status-react/blob/1bf68094e379c13cb72f33a48571ff664264ee9d/src/status_im/ethereum/json_rpc.cljs
• Whisper/Waku extensions https://github.com/status-im/status-go/tree/79b8112f89e218e5d24b6c4400eeaf6d4e8ec6db/services/ext
• Map of components https://miro.com/app/board/o9J_kuuReug=/
• Waku v2 sketch https://miro.com/app/board/o9J_kqPBRbg=/

[oskarth]

Part of wish list for Waku v2, enabling this kind of use case. Once we are on libp2p we can experiment with enabling this kind of ting: status-im/specs#117

[oskarth]

See vacp2p/rfc#153 for some things that need to be done spec/impl wise