Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update WakuPeerStore to store pubSubTopics for a peer #734

Merged
merged 6 commits into from
Sep 14, 2023

Conversation

chaitanyaprem
Copy link
Collaborator

@chaitanyaprem chaitanyaprem commented Sep 14, 2023

Description

Store pubSubTopics that a peer supports. This is done keeping static sharding in mind as per #727
For autosharding, this can be reused, whereas API updates would be requried to only accept contentTopics.

Will take up sharded peer-mgmt in a separate PR in order to avoid making this one too big :)

Changes

  • Update WakuPeerStoreImpl to provide additional methods to set, retrieve pubSubTopics for a peer
  • AddPeer shall take an optional list of pubSubTopics. If nothing is passed, ENR is used to derive topics for relaySharding. If ENR is not present/empty for the peer then defaultRelayTopic is set.
  • Static Peers are provided with all pubSubTopics passed in the config as well as derived from content-topics(in case of auto-sharding). This means configuration should be changed to make it more fine-grain so that a static-Node can be specified which topics it supports.
  • Peers discovered with discv5 , PeerExchange, DNSDiscovery shall have ENR based on which pubSubTopics are derived.
  • Provide API's to fetch peers based on pubSubTopic

Tests

All unit tests have passed.
All builds have passed.

Note: Did not add new tests for these, as existing tests for discovery protocols seem to cover all flows.

@chaitanyaprem chaitanyaprem changed the title feat: update peerStore to store pubSubTopics a peer is linked to feat: update WakuPeerStore to store pubSubTopics a peer is linked to Sep 14, 2023
@chaitanyaprem chaitanyaprem self-assigned this Sep 14, 2023
@status-im-auto
Copy link

status-im-auto commented Sep 14, 2023

Jenkins Builds

Click to see older builds (24)
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 2d136a6 #1 2023-09-14 06:06:13 ~2 min nix-flake 📄log
✔️ 2d136a6 #1 2023-09-14 06:06:37 ~2 min linux 📦deb
✔️ 2d136a6 #1 2023-09-14 06:07:11 ~3 min tests 📄log
✔️ 2d136a6 #1 2023-09-14 06:07:16 ~3 min tests 📄log
✔️ 2d136a6 #1 2023-09-14 06:08:18 ~4 min android 📦tgz
✔️ 2d136a6 #1 2023-09-14 06:08:54 ~4 min ios 📦tgz
✔️ eca4a8e #2 2023-09-14 06:28:15 ~55 sec linux 📦deb
✔️ eca4a8e #2 2023-09-14 06:28:42 ~1 min tests 📄log
✔️ eca4a8e #2 2023-09-14 06:29:13 ~1 min tests 📄log
✔️ eca4a8e #2 2023-09-14 06:29:19 ~1 min nix-flake 📄log
✔️ eca4a8e #2 2023-09-14 06:30:50 ~3 min android 📦tgz
✔️ eca4a8e #2 2023-09-14 06:30:52 ~3 min ios 📦tgz
✔️ 22359a6 #3 2023-09-14 07:33:22 ~57 sec linux 📦deb
✔️ 22359a6 #3 2023-09-14 07:34:19 ~1 min nix-flake 📄log
✔️ 22359a6 #3 2023-09-14 07:34:24 ~1 min tests 📄log
✔️ 22359a6 #3 2023-09-14 07:35:16 ~2 min tests 📄log
✔️ 22359a6 #3 2023-09-14 07:35:48 ~3 min android 📦tgz
✔️ 22359a6 #3 2023-09-14 07:36:06 ~3 min ios 📦tgz
✔️ 9f19a38 #4 2023-09-14 09:10:10 ~54 sec linux 📦deb
✔️ 9f19a38 #4 2023-09-14 09:10:42 ~1 min tests 📄log
✔️ 9f19a38 #4 2023-09-14 09:11:08 ~1 min tests 📄log
✔️ 9f19a38 #4 2023-09-14 09:11:09 ~1 min nix-flake 📄log
✔️ 9f19a38 #4 2023-09-14 09:12:37 ~3 min android 📦tgz
✔️ 9f19a38 #4 2023-09-14 09:13:32 ~4 min ios 📦tgz
Commit #️⃣ Finished (UTC) Duration Platform Result
✔️ 6f8933e #5 2023-09-14 10:38:18 ~54 sec tests 📄log
✔️ 6f8933e #5 2023-09-14 10:38:22 ~57 sec linux 📦deb
✔️ 6f8933e #5 2023-09-14 10:39:10 ~1 min tests 📄log
✔️ 6f8933e #5 2023-09-14 10:39:18 ~1 min nix-flake 📄log
✔️ 6f8933e #5 2023-09-14 10:40:30 ~3 min android 📦tgz
✔️ 6f8933e #5 2023-09-14 10:40:51 ~3 min ios 📦tgz
✔️ 3d68626 #6 2023-09-14 14:50:16 ~1 min nix-flake 📄log
✔️ 3d68626 #6 2023-09-14 14:51:20 ~2 min tests 📄log
✔️ 3d68626 #6 2023-09-14 14:51:32 ~3 min tests 📄log
✔️ 3d68626 #6 2023-09-14 14:51:52 ~3 min android 📦tgz
✔️ 3d68626 #6 2023-09-14 14:52:00 ~3 min ios 📦tgz
✔️ 3d68626 #6 2023-09-14 14:53:37 ~5 min linux 📦deb

@chaitanyaprem chaitanyaprem changed the title feat: update WakuPeerStore to store pubSubTopics a peer is linked to feat: update WakuPeerStore to store pubSubTopics for a peer Sep 14, 2023
@chaitanyaprem chaitanyaprem marked this pull request as ready for review September 14, 2023 07:32
Copy link
Member

@richard-ramos richard-ramos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, however we'll probably need to add code in future PRs to maintain the peerstore, as peers might join/leave the pubsub topic. Looking at go-libp2p-pubsub code, I see there are some PeerJoin / PeerLeave events we can probably subscribe to, and this way determine whether a peer still is part of a pubsub topic or not

@chaitanyaprem
Copy link
Collaborator Author

LGTM, however we'll probably need to add code in future PRs to maintain the peerstore, as peers might join/leave the pubsub topic. Looking at go-libp2p-pubsub code, I see there are some PeerJoin / PeerLeave events we can probably subscribe to, and this way determine whether a peer still is part of a pubsub topic or not

Right, that also needs to be handled.
Let me add this as a task in the parent issue to keep track of this.

@chaitanyaprem chaitanyaprem merged commit bfc3083 into master Sep 14, 2023
@chaitanyaprem chaitanyaprem deleted the feat/static-shard-peer-mgmt branch September 14, 2023 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants