feat: Deprecate Named Sharding and Update Lightpush Client API

[chair28980]

In the context of discussions around named sharding and peer management for the Waku network, it is proposed to make the following changes to the Waku Network:

1. Deprecate Support for Named Sharding:

• It is recommended to remove support for named sharding.
• Peer management should ensure that only peers supporting sharded topics are connected.

2. Lightpush Protocol Updates:

• The lightpush protocol itself should never be aware of the sharded format of pubsub topics. It is an extension of libp2p's gossipsub.
• On a protocol level, the lightpush service node needs to handle requests on unsupported pubsub topics by returning a clear error code.

3. Lightpush Client API Changes:

• To maintain adherence to the sharded format, the lightpush client API should be updated.
• New proposed format: lightPublish(WakuMessage, ClusterID, ShardNumber)
• For legacy requests on named topics, the API could use a trial and error process to send the request to random service nodes. This can be done using the format: lightPublish(WakuMessage, PubsubTopic)
• A request format that bypasses our peer management/selection: lightPublish(WakuMessage, PubsubTopic, ServiceNodeMultiaddr)

---

• Users of custom pubsub topics should select a cluster id.
• Use static sharding until the launch of The Waku Network.
• Default Pubsub topic support remains for now, to be phased out as a later step

Acceptance Criteria:

• Remove support for named sharding (except for default pubsub topic)
• Update lightpush protocol to handle unsupported pubsub topics with clear error codes.
• Introduce the new lightpush client API as mentioned above.
• Update documentation to reflect these changes and guide users accordingly.

[harsh-98]

Weekly update

• achieved: Create PR for review for removing of Named Pubsubtopic.

[harsh-98]

@chair28980 regarding Peer management should ensure that only peers supporting sharded topics are connected.. This also includes default pubsub topic right?

[chair28980]

@harsh-98 good question, I'm not sure, @fryorcraken / @jm-clius your thoughts on this?

[fryorcraken]

@chair28980 regarding Peer management should ensure that only peers supporting sharded topics are connected.. This also includes default pubsub topic right?

Cc @jm-clius @SionoiS on this

[SionoiS]

It should IMO but it's going to break so much stuff... Necessary pain I guess.

If we don't remove the default pubsub topic then what's the point of dropping support for named shard?

[jm-clius]

Indeed, although we should certainly phase this in: I think at minimum we need the public Waku Network running, change default configuration to use this cluster of shards and then we can consider deprecating this.

[fryorcraken]

IMO default pubsub topic should be the only supported named shard moving forward.
Best to keep compatibility at first, until we are happy to push for migration of current users to autosharding.

In terms of behaviour, I'd say in general "no shard" = default pubsub topic.

[chair28980]

@harsh-98 heads up we are iceboxing all E:1.4 issues that are still in-progress including this issue.

[chaitanyaprem]

Descoping this for go-waku as we already support below in all APi's

• no shard means default pubsubTopic