nwaku for operators trial

[jm-clius]

Background

One of the main takeaways from the Q4 2022 Vac Sustainability and Business Workshop was the need for nwaku to settle into a niche of being an easy-to-install, easy-to-monitor, lightweight and robust option for user-run Waku v2 nodes.

Currently the main obstacles to achieving this goal is:

• incomplete or unclear documentation for users on how to set up and dimension their own nwaku nodes for different protocols
• limited pre-compiled binaries for easy setup (users usually need to clone the repo and build nwaku themselves, seldomly a smooth process)
• limited insight into actual robustness of running a nwaku node in the long term, especially for lesser-used protocols, such as filter and lightpush
• limited visibility (e.g. a metric) of the number of user-run nodes in the default Waku v2 network and number of applications using Waku v2
• lack of ambient peer discovery

What do we want to achieve as a first step?

This issue tracks the work necessary to launch a Vac-internal operator-run node trial. Specifically, we want:

Goal 1: Team-internal operators set up own nwaku nodes

Operators to follow instructions to set up their own nwaku node using a precompiled binary that:

• participates and is discoverable in the default relay network
• provides a dimensioned store
• provide lightpush and filter capabilities

Between team members we could probably run a hybrid network of nodes on at least Windows, Linux and MacOS.

Goal 2: Operators monitor their nodes

Operators to follow instructions to monitor their nodes, e.g. using an included metrics dashboard, looking at logs, etc.

Goal 3: Operators to provide informal, weekly updates

This should track two main qualitative metrics:

1. "How easy is it to operate a node?"

• feedback on documentation/instructions. What are the gaps in knowledge that should be bridged/obscured from end users?
• are the included tooling enough for operation?
• feedback re available metrics/logs and their usefulness
• feedback re metrics dashboard

2. "How well does the node run?"

• i.e. how stable has it been running as far as you can tell from metrics/logs
• how many stable connections does the node maintain
• how many new, useful peers has the node discovered (only PX during trial)

As a team we can use the various chat2 clients to simulate different scenarios during the trial, to ensure we properly test filter, lightpush, etc.

What do we have already?

• Some scattered documentation on how to build and run a nwaku node from source code with default config, e.g. here and here
• A basic Dockerfile to build a deployable image
• Ubuntu and MacOS binaries included as release assets

What still needs to be done?

At minimum, before launching trial:

• Distributable nwaku binaries available for all major OSs and architectures: Automate release and build distributable binaries #611
• Published all-in-one docker image, including some tooling for monitoring: Publish docker image for wakunode2 and document its usage #737
• Minimal ambient peer discovery: GossipSub peer exchange: Gossipsub peer exchange vacp2p/nim-libp2p#647
• Basic documentation on how to run and monitor a nwaku node from scratch. We could use the The Nimbus Book as inspiration here.
• Operator-focused metrics (e.g. number of content topics in use, current message rate, updated dashboards, etc.)

Nice to haves before trial launch:

• Proper nwaku resource management: nim-waku external files and resources #798 (especially if we're going to add more resources, such as a Grafana dashboard)
• Ability to mark messages as "do not store": 13/WAKU2-STORE / 14/WAKU2-MESSAGE: Allow messages to be marked as "don't store" vacp2p/rfc#441
• Support for DNS addresses: Does not support dns4 multiaddr #521
• Solution for the store timestamping and indexing issue, under discussion here
• Document nodekey generation: Doc: Document how to generate and store a nodekey #847

cc @oskarth @staheri14 @kaiserd @D4nte

[jm-clius]

May overlap with some goals in #827 as well.

[D4nte]

I'd add some documentation to use a letsencrypt certificate to secure the websocket connection.

[oskarth]

Great breakdown, thank you!

I wonder if we need precompiled binaries for the first step? I don't think Nimbus provided this in the initially stages (I still recompile it every release for my beacon chain node)

[jm-clius]

Changed milestone to Release 0.11 as we require people to run the fixes in 0.10 for an effective trial.

[jm-clius]

Operator trial successfully launched. Feedback being tracked in #1208