PubSub topic peer discovery

[akru]

Motivation

I happy to see rust implementation support Gossipsub protocol, but what’s about topic peer discovery?

[mxinden]

You find some information in the GossipSub v1.0 specification e.g. using a DHT like Kademlia for discovery. In addition the GossipSub v1.1 specification mentions PeerExchange which is an internal logic to tell peers who to connect to.

Pinging @AgeManning here as well in case I am missing something.

[akru]

Go implementation use active peer discovery (https://github.com/libp2p/go-libp2p-discovery), is it planned to implement for rust-libp2p?

[tomaka]

That repository is just an abstraction. rust-libp2p doesn't distinguish discovery mechanisms from non-discovery mechanisms. Everything goes through the NetworkBehaviour trait that libp2p-mdns and libp2p-kad (the two discovery mechanisms in this repo) implement.

[akru]

@tomaka what do you think about implementing gossipsub peer discovery/exchange for rust-libp2p? I'm really interested in it for my project. Maybe I can get this task.

[mxinden]

@akru in case you are referring to the above mentioned PeerExchange mechanism of GossipSub v1.1 as far as I know @AgeManning is working on v1.1, thus best to coordinate with @AgeManning.

[akru]

Got it, thank you @mxinden!

[AgeManning]

Hey @akru

As @mxinden points out there is a gossipsub enhancement that allows gossipsub peer discovery for topics. If you already know of at least one peer on a topic, that peer now has the ability to tell you about other peers on that topic. You'd still need a small collection of peers to bootstrap off.

Typically applications use an active discovery mechanism to find and connect to live peers, like the ones in rust-libp2p @tomaka has pointed out.

I have gossipsub 1.1 on my to-do list. I'm yet to start, but should hopefully get there in the next week or two.

Is the peer exchange in 1.1 explicitly what you are chasing? Or just for a discovery mechanism that can discover peers on specific topics?

[akru]

Hello @AgeManning!

Thank you for detailed explanation.

If I correct understand you now is possible to get list of peers for specified topic from another peer whom already connected on. Is this behavior already implemented in libp2p?

Ideally, I wonder to just bootstrap by static node set (that could not connected to my personal topic) and find other peers whom listen or publish data into my topic in the network. Probably it could be reached by peer exchange. What do you think?

[AgeManning]

If I correct understand you now is possible to get list of peers for specified topic from another peer whom already connected on. Is this behavior already implemented in libp2p?

This is not currently implemented in rust-libp2p gossipsub. It is implemented in go-libp2p gossipsub. The idea is that if you are connected (or try to connect) to a node on a topic and that node has too many peers or is leaving, it can pass you on a list of peers for that topic. I've not yet decided how this looks to the user, as to whether we tell the user new peers are found, or automatically dial and maintain a strong connection to the topic where possible.

Ideally, I wonder to just bootstrap by static node set (that could not connected to my personal topic) and find other peers whom listen or publish data into my topic in the network. Probably it could be reached by peer exchange. What do you think?

If i understand you correctly, you are suggesting a static node list of peers that are not necessarily on the topic you are chasing right? I imagine this wouldn't be great. I've not yet looked into the implementation details around using gossipsub for a discovery mechanism. Potentially you can try and graft to each peer on your topic and they should respond with any peers they know about on your topic (but I'm not sure the Peer exchange is actually designed for what you are after (I could be wrong tho)). Once I've built it, I'll be in a better position to answer this.

In the meantime, rust-libp2p offers kademlia. If you use the kademlia protocol, you can discover nodes on your network, when connecting to them, gossipsub sends topic subscriptions, so you could identify nodes on your topic via finding peers using kademlia.

[romanb]

Since gossipsub 1.1 has landed and the discussion is a bit dated, I'm closing. Feel free to open new issues or discussions (or re-open this issue to continue this discussion) as necessary.