feat: opt-in spam protected topic with signed messages

[alrevuelta]

Problem

Publishing messages to gossipsub topics has no limits, meaning that anyone can send messages at a high rate and DoS the network. This would elevate the bandwidth consumtion of all nodes subscribed to said pubsub topic, making it prohivitive (in terms of bandwidth) to be subscribed to it.

Suggested solution

This issue tracks a solution where certain gossipsub topics can be configured to only accept messages signed with a given key, that only trusted entities know. This key can be pre-shared among a set of participants, that are trusted to make a fair usage of the network, publishing messages at a reasonable rate/size. Note that this key can be shared/reused among multiple participants. Note that this is an opt-in solution that operators can chose to deploy in their topics, but its not enforced in the default pubsub topics that waku has.

Note also that a node only requires the public keys to validate messages, so that participation can be kept open. Only private key holders will be able to sign messages and get the gossiped in the network.

Summary of the solution:

• Make spam protected topics configurable: [pubsub-topic-name, public-key] via cli flags.
• Don't break any backwards compatibility with existing non protected topics, which should still allow all traffic (eg default pubsub topic)
• Add a new optional signature message field to include the signature of the message hash.
• All messages whose signatures weren't produced by public-key for the configured topic-name will be rejected.
• A peer sending multiple invalid messages for said topics, gets its reputation lowered and eventually is disconnected from.

Acceptance criteria

• Implement libp2p message validation for the configured topics, only accepting messaes signed with configured key. feat: dos protected topic relay msgs based on meta field #1614
• Add configuration via cli + metrics: feat: configure protected topics via cli #1696
• Add metrics: chore: add validator for dos protec metrics and move to app #1704
• Roll out the solution to our fleets, segregating nodes containing non protected topic.

[LNSD]

I find the problem description too broad.

Could you add to the problem description a specific use case that you are trying to cover with this solution? (e.g., the status community scenario or a "generic group of nodes subscribed to a pub-sub topic that share a group key pair").

I also miss a list of assumptions (e.g., a pre-shared private-public key pair) that the solution relies on. Could you add it too?

[alrevuelta]

@LNSD made some claritications, hope it helps.

[LNSD]

Note also that a node only requires the public keys to validate messages, so that participation can be kept open. Only private key holders will be able to sign messages and get the gossiped in the network.

• How do you plan to generate/distribute/exchange this key pair?
• What node roles do you envision in this solution? (e.g., relay nodes that have the public key will be able to DoS protect the pub-sub topic but not publish messages, only nodes that have the private key will be able to publish messages to the pub-sub network)

Could you also add this information to the description?

[LNSD]

Note that this is an opt-in solution that operators can chose to deploy in their topics, but its not enforced in the default pubsub topics that waku has.

As an Opt-in solution, does this functionality require a relayer node to be rebooted with a different configuration? Are you thinking of providing also a runtime configuration functionality (we are supporting relay subscriptions on runtime)

[LNSD]

• Make spam protected topics configurable: [topic-name, public-key] via cli flags.

I guess that with topic-name you are referring to "pub-sub" topic, right? As it is the "distribution layer" topic. Could you clarify it?

[rymnc]

Note also that a node only requires the public keys to validate messages, so that participation can be kept open. Only private key holders will be able to sign messages and get the gossiped in the network.

• How do you plan to generate/distribute/exchange this key pair?
• What node roles do you envision in this solution? (e.g., relay nodes that have the public key will be able to DoS protect the pub-sub topic but not publish messages, only nodes that have the private key will be able to publish messages to the pub-sub network)

re: distribution of this keypair, could we use the shared key of the community members? this way, we leave the generation of the keypair to the key-exchange mechanism, and community nodes can then ignore messages that don't belong to the community.

[LNSD]

• Add a new optional signature message field to include the signature of the message hash.

What do you mean by signature? What cryptographic schema are you thinking to use as part of the solution?

Note that the schema should be simple enough so it can be performed by the relay nodes during the message validation. The size of the "signature" may be also important.

[LNSD]

re: distribution of this keypair, could we use the shared key of the community members? this way, we leave the generation of the keypair to the key-exchange mechanism, and community nodes can then ignore messages that don't belong to the community.

I am "ok" with that for the "status specific" scenario/use case, but... If a different application (not status) wants to use this DoS protection mechanism. Does Waku provide a "key distribution" mechanism to share the key pair? Shall they implement/use the status specification for key exchange?

In the end, what is the scope of this development/solution?

[rymnc]

@LNSD yeah, lets leave it up to the applications to share the key material out of band. They can use either
https://rfc.vac.dev/spec/35/ or https://rfc.vac.dev/spec/53 to perform a key-exchange, and generation of the shared key. There should probably be some documentation alluding to it though.

[LNSD]

@LNSD yeah, lets leave it up to the applications to share the key material out of band. They can use either rfc.vac.dev/spec/35 or rfc.vac.dev/spec/53 to perform a key-exchange, and generation of the shared key. There should probably be some documentation alluding to it though.

In that case, @alrevuelta, could you specify this "out-of-band pre-shared keys distribution" assumption in the issue description?

I have more questions about "how do you provide these pre-shared keys to the nodes", but I need first the "node roles" question to be answered.

[alrevuelta]

How do you plan to generate/distribute/exchange this key pair?

Each operator is free to chose the mechanism they prefer to do all three. Not defined by waku.

What node roles do you envision in this solution? (e.g., relay nodes that have the public key will be able to DoS protect the pub-sub topic but not publish messages, only nodes that have the private key will be able to publish messages to the pub-sub network)

With this feature I can think of two roles, which only applies to DoS protected topics:

• nwaku relay: Is configured with a given key in a given pubsub topic. Is DoS protected in this topic and can relay messages (check it validity using the signature)
• publisher: A node that knows the private key and can send valid messages (signed) that are propagated ok in a given pubsub topic. I would not link this role with nwaku.

As an Opt-in solution, does this functionality require a relayer node to be rebooted with a different configuration? Are you thinking of providing also a runtime configuration functionality (we are supporting relay subscriptions on runtime)

Yes, this will require a new configuration, probably with a new cli flag and ofc rebooting. The new configuration would be the gossipsub topic + the public key used to validate in that topic.

Regarding runtime configuration, to be dicussed. Perhaps we can hardcode the status public key in the code, so that anyone can relay traffic in that gossipsub topic without having to manually configure it. No strong opinion on it.

I guess that with topic-name you are referring to "pub-sub" topic, right? As it is the "distribution layer" topic. Could you clarify it?

Yes, fixed.

re: distribution of this keypair, could we use the shared key of the community members? this way, we leave the generation of the keypair to the key-exchange mechanism, and community nodes can then ignore messages that don't belong to the community

Sure, the key here can be freely chosen by operators in the gossip sub topics, as long as its a secp256k1 keypair.

What do you mean by signature? What cryptographic schema are you thinking to use as part of the solution?

The signature is the message hash signed with a secp256k1 private key, that can be verified by its public key equivalent. Note that its the same as many blockchains, which can be useful (tooling, reusing addresses, etc).

Does Waku provide a "key distribution" mechanism to share the key pair?

Nop, all this is done by node operators wanting to run this protected gossipsub topic, and they are responsible of sharing it carefuly with trusted users.

In that case, @alrevuelta, could you specify this "out-of-band pre-shared keys distribution" assumption in the issue description?

Sure.

I have more questions about "how do you provide these pre-shared keys to the nodes", but I need first the "node roles" question to be answered.

• From an nwaku operator point of view, the only role would be relayer. The public pre-shared key would be provided as a cli argument mapping the public key and gossipsub topic. Perhaps --dos-gossipsub-topic=gossipsub-topic/public-key, but open for different names and ways.
• On the other hand, related to the publisher role, an app using waku (eg go-waku/status) wanting to publish in said topic would need to have the private key embeeded in the code (but hidden) so that this node can publish valid signed messages. No implementation details here besides treating the private key as private.

@LNSD thanks for the comments, really rellevant. Replied inline. Will update the issue

[LNSD]

Based on this response:

With this feature I can think of two roles, which only applies to DoS protected topics:

• nwaku relay: Is configured with a given key in a given pubsub topic. Is DoS protected in this topic and can relay messages (check it validity using the signature)
• publisher: A node that knows the private key and can send valid messages (signed) that are propagated ok in a given pubsub topic. I would not link this role with nwaku.

And this response:

• From an nwaku operator point of view, the only role would be relayer. The public pre-shared key would be provided as a cli argument mapping the public key and gossipsub topic. Perhaps --dos-gossipsub-topic=gossipsub-topic/public-key, but open for different names and ways.
• On the other hand, related to the publisher role, an app using waku (eg go-waku/status) wanting to publish in said topic would need to have the private key embeeded in the code (but hidden) so that this node can publish valid signed messages. No implementation details here besides treating the private key as private.

And this other response:

Does Waku provide a "key distribution" mechanism to share the key pair?

Nop, all this is done by node operators wanting to run this protected gossipsub topic, and they are responsible of sharing it carefuly with trusted users.

I understand that you envision the network operator having to configure the fleet nodes for each community that wants to use that node as a relayer. This means that the Status CC community pre-shared key will have to be manually configured to the status production fleet nodes by Jakub.

This approach has a clear limitation from my point of view: What happens when the "common pre-shared key" is invalidated (e.g., a Status community member is removed, and a new community key is generated)?

Additionally, hardcoding or embedding any keys inside the code will bring many problems in the future. We need to find a different strategy.

[LNSD]

What do you mean by signature? What cryptographic schema are you thinking to use as part of the solution?

The signature is the message hash signed with a secp256k1 private key, that can be verified by its public key equivalent. Note that its the same as many blockchains, which can be useful (tooling, reusing addresses, etc).

I imagined that you were thinking of signing using the preshared private key, but my question was related to: What and how are you going to sign the message?

Could you provide a detailed explanation of the signature schema that you want to use to sign the message?

[jm-clius]

@LNSD this is indeed a simple scheme, but has been discussed with Status Communities to understand the alternatives vs doing something which will work in MVP timeframe, even if it has limitations.

Most of the limitations of this approach indeed comes from the manual approach that's necessary for infrastructure nodes. Assumptions are:

• the community provides its own infra nodes (even if we do it on their behalf, those infra nodes are seen as "belonging" to the community)
• the community can provide the public key for validation to the infra nodes. Their infra nodes get configured with this to validate the application (in other words the infra nodes run something of an application on top of the general protocols).

What happens when the "common pre-shared key" is invalidated (e.g., a Status community member is removed, and a new community key is generated)?

Good point, but this should not happen often, at least within MVP timeframe. The assumption is that there exists a public key material that Community signatures can be validated with. Can cc @cammellos here again for confirmation of this requirement.

Note that general DoS protection does not require that infra nodes perform the validation as well - if all community members perform the validation and the proportion of infra nodes are relatively small, we'll still get some DoS protection. But it's much better to have infra perform validation too and the application validation logic should be so simple that they could.

What and how are you going to sign the message? Could you provide a detailed explanation of the signature schema that you want to use to sign the message

Waku (app) only validates the message against the shared public key. It should not sign messages. Messages arrived signed from the application and only the application can produce and sign messages, as it has access to the shared private key.

[rymnc]

Maybe the validator should be controlled by the rest api instead of cli arguments?
i.e whenever the pubkey of the community changes, the status app sends a request to the waku node to update the validator to use the new pubkey (I can see some messages being incorrectly invalidated based on how long it takes the app to receive the new pubkey, and make the appropriate change to the validator)

[alrevuelta]

I understand that you envision the network operator having to configure the fleet nodes for each community that wants to use that node as a relayer

Yes, and I don't think there is any other way. Even if its configured via RPC, it involves some action. But note that its an opt-in feature to prevent some DoS. Meaning it is backwards compatible with nodes not validating messages in the same topic.

What happens when the "common pre-shared key" is invalidated (e.g., a Status community member is removed, and a new community key is generated)?

This will ofc require changing the configuration, with some manual intervention.

I imagined that you were thinking of signing using the preshared private key, but my question was related to: What and how are you going to sign the message?

what: message digest (hash of the message)
how: see ecdsa signatures using secp256k1 curve.

Additionally, hardcoding or embedding any keys inside the code will bring many problems in the future. We need to find a different strategy.

Lets leave this by now then.

Maybe the validator should be controlled by the rest api instead of cli arguments?
i.e whenever the pubkey of the community changes, the status app sends a request to the waku node to update the validator to use the new pubkey (I can see some messages being incorrectly invalidated based on how long it takes the app to receive the new pubkey, and make the appropriate change to the validator)

I see some added complexity here. eg when you start the node it wont be protected until its configured via a RPC call. I also think its very prone to missconfiguration.

[jm-clius]

Implemented and tested in nwaku. Interop with go-waku also tested. Next steps is integration in status-go and community. cc @cammellos @richard-ramos