feat: discovery & peer management for static shards

[richard-ramos]

Problem

Work is in progress in status-go to integrate go-waku's rendezvous, but it will run into a blocking issue: we noticed there is even tho you can advertise the pubsubtopic/shards you are subscribed to, and ask for peers from those shards, you cannot later retrieve and connect to those peers as information about the pubsub topics supported by the nodes is not stored.

Something else, mentioned by @chaitanyaprem during today's go-waku PM call is that we need to ensure there's at least D peers connected per shard.

Note: This issue tracks the basic Peer-Management updates related to sharding (static/auto). More advanced features like peer-scoring etc would be handled as part of separate issue. This is to reduce scope and iterate.

Tasks

• feat: update WakuPeerStore to store pubSubTopics for a peer #734
• Feat : handle dynamic peer topic sub unsub #751
• feat: Sharded peer management - Relay #764
• Add tests to validate specific functionality

Service Peer Management for static shards is being handled by #769

[chaitanyaprem]

@richard-ramos , I started working on storing pubSubTopic(s) a peer is participating in as part of peerMetaData. But then i had noticed that go-libp2p-pubSub is having this information wrt peers which are connected. We already seem to be using it with rendezvous

go-waku/cmd/waku/node.go

Line 365 in 4b1c188

peerCnt := len(wakuNode.Relay().PubSub().ListPeers(nodeTopic))

Do you still see the need to store this info additionally as part of peerStore? One limitation i see with using pubSub's information is, i think only the actively connected peers are maintained and peers to whom connections have been dropped are not maintained.

[richard-ramos]

Hm. I see. Good point. Perhaps the focus for this task should be then on having the peer connector keep track of the number of peers connected on the pubsub topics that the waku node is currently subscribed to so we can guarantee there's at least D peers connected for each pubsub topic and once that number is achieved not connect to any additional peer?

I do find the idea of adding the pubsub topic(s) of a discovered peer in here: https://github.com/waku-org/go-waku/blob/master/waku/v2/peermanager/peer_connector.go#L29-L33 so the peer connector can take an informed decision on whether is it worth it to connect to that peer or not.

---

@ilmotta, perhaps this code will help you with integrating rendezvous in status-go

go-waku/cmd/waku/node.go

Lines 355 to 380 in 4b1c188

	go func(nodeTopic string) {
	defer wg.Done()
	desiredOutDegree := wakuNode.Relay().Params().D
	t := time.NewTicker(7 * time.Second)
	defer t.Stop()
	for {
	select {
	case <-ctx.Done():
	return
	case <-t.C:
	peerCnt := len(wakuNode.Relay().PubSub().ListPeers(nodeTopic))
	peersToFind := desiredOutDegree - peerCnt
	if peersToFind <= 0 {
	continue
	}
	rp := <-iter.Next(ctx)
	if rp == nil {
	continue
	}
	ctx, cancel := context.WithTimeout(ctx, 7*time.Second)
	wakuNode.Rendezvous().DiscoverWithNamespace(ctx, nodeTopic, rp, peersToFind)
	cancel()
	}
	}
	}(nodeTopic)

[chaitanyaprem]

Hm. I see. Good point. Perhaps the focus for this task should be then on having the peer connector keep track of the number of peers connected on the pubsub topics that the waku node is currently subscribed to so we can guarantee there's at least D peers connected for each pubsub topic and once that number is achieved not connect to any additional peer?

Yeah, but in order to achieve this looks like we will have to store the pubSubTopics as metadata in peerStore. Let's say peerconnector identifies there are less peers for a particular topic, then it has to look into the peerStore for peers which are disconnected and availble to connect for that topic. If it cannot find, then anyways some sort of discovery has to be triggered to find more.

But atleast since status-go would be interested in peers that are already connected for a topic the above approach should be fine. I am wondering if it would be good to provide a utility API which takes a topic and returns this info, rather than directly exposing the pubSub interface. Other applications would also benefit from this.

I do find the idea of adding the pubsub topic(s) of a discovered peer in here: https://github.com/waku-org/go-waku/blob/master/waku/v2/peermanager/peer_connector.go#L29-L33 so the peer connector can take an informed decision on whether is it worth it to connect to that peer or not.

Yes

[chaitanyaprem]

Will be following this approach for sharded peer-mgmt as per waku-org/pm#67. Will come up with an MVP along with @SionoiS and implement it so the same can be used for static and auto-sharding.

[chaitanyaprem]

Weekly Update

• achieved: Update WakuPeerStore to store pubSubTopics for a peer.
• next: Sharded Peer Management considering static sharding for Status communities.

[chaitanyaprem]

Weekly Update

• achieved: handle dynamic topic sub/unsub and update peerMetadata.
• next: relay peer mgmt for static/auto sharding

[chaitanyaprem]

Weekly Update

• achieved: basic relay peer mgmt for static/auto sharding