Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Deprecate Named Sharding and Update Lightpush Client API #2163

Closed
4 tasks
chair28980 opened this issue Oct 26, 2023 · 1 comment
Closed
4 tasks

feat: Deprecate Named Sharding and Update Lightpush Client API #2163

chair28980 opened this issue Oct 26, 2023 · 1 comment
Assignees
Labels
effort/weeks Estimated to be completed in a few weeks

Comments

@chair28980
Copy link
Contributor

chair28980 commented Oct 26, 2023

In the context of discussions around named sharding and peer management for the Waku network, it is proposed to make the following changes to the Waku Network:

  1. Deprecate Support for Named Sharding:
  • It is recommended to remove support for named sharding.
  • Peer management should ensure that only peers supporting sharded topics are connected.
  1. Lightpush Protocol Updates:
  • The lightpush protocol itself should never be aware of the sharded format of pubsub topics. It is an extension of libp2p's gossipsub.
  • On a protocol level, the lightpush service node needs to handle requests on unsupported pubsub topics by returning a clear error code.
  1. Lightpush Client API Changes:
  • To maintain adherence to the sharded format, the lightpush client API should be updated.
  • New proposed format: lightPublish(WakuMessage, ClusterID, ShardNumber)
  • For legacy requests on named topics, the API could use a trial and error process to send the request to random service nodes. This can be done using the format: lightPublish(WakuMessage, PubsubTopic)
  • A request format that bypasses our peer management/selection: lightPublish(WakuMessage, PubsubTopic, ServiceNodeMultiaddr)

  • Users of custom pubsub topics should select a cluster id.
  • Use static sharding until the launch of The Waku Network.
  • Default Pubsub topic support remains for now, to be phased out as a later step

Acceptance Criteria:

  • Remove support for named sharding (except for default pubsub topic)
  • Update lightpush protocol to handle unsupported pubsub topics with clear error codes.
  • Introduce the new lightpush client API as mentioned above.
  • Update documentation to reflect these changes and guide users accordingly.
@chair28980 chair28980 added the E:1.4: Sharded peer management and discovery See https://github.com/waku-org/pm/issues/67 for details label Oct 26, 2023
@SionoiS SionoiS self-assigned this Oct 27, 2023
@chair28980 chair28980 moved this to Triage in Waku Oct 31, 2023
@chair28980 chair28980 removed the status in Waku Oct 31, 2023
@chair28980 chair28980 moved this to To Do in Waku Oct 31, 2023
@chair28980 chair28980 removed the E:1.4: Sharded peer management and discovery See https://github.com/waku-org/pm/issues/67 for details label Dec 5, 2023
@SionoiS SionoiS removed their assignment Jan 26, 2024
@gabrielmer gabrielmer self-assigned this May 23, 2024
@gabrielmer gabrielmer moved this from To Do to In Progress in Waku May 23, 2024
@gabrielmer gabrielmer added effort/weeks Estimated to be completed in a few weeks and removed effort/weeks Estimated to be completed in a few weeks labels Jun 4, 2024
@gabrielmer
Copy link
Contributor

Separating this issue into its 3 components:

  1. chore: deprecate support for named sharding: #2772
  2. chore: update lightpush protocol to handle unsupported pubsub topics with clear error codes #2773
  3. chore: update lightpush client API #2774

Closing this one and will track the issues separately. Some of them might not be relevant anymore

@github-project-automation github-project-automation bot moved this from In Progress to Done in Waku Jun 4, 2024
@gabrielmer gabrielmer closed this as not planned Won't fix, can't repro, duplicate, stale Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/weeks Estimated to be completed in a few weeks
Projects
Archived in project
Development

No branches or pull requests

3 participants