[Deliverable] Store v3-beta - Message Hashes

[chair28980]

Project: https://github.com/orgs/waku-org/projects/16/views/1

Epics

• [Epic: research] Store v3-beta - Message Hashes research#103
• [Epic: nwaku] Store v3-beta - Message Hashes nwaku#3070
• [Epic: QA] Store v3-beta - Message Hashes #225
• [Epic: go-waku] Store v3-beta - Message Hashes  go-waku#1229

Research Milestone

Output

Spec published here: https://github.com/waku-org/specs/blob/master/standards/core/store.md
Reference implementation: waku-org/nwaku#2431
and released in https://github.com/waku-org/nwaku/releases/tag/v0.28.0

Summary

• Goal: After this upgrade, the network will provide distributed and synchronised store services.
• Deliverables: (2), (3)
• Research tracks: Message Reliability
• Estimated date of completion: 2024Q2

An improved version of the Store protocol, marking a crucial increment towards a synchronisation protocol:

• introduces the concept of deterministic message hashes to index messages
• considers the Store as a key-value store
• allows for querying a list of keys (message hashes) from the Store
• allows for querying for the full message content (values) of a set of keys from the Store
• keeps all previous value-based filtering (e.g. content topic, timestamp) in place

The proposed PR to simplify the Store protocol and use message
hashes as index/cursor, can be used as a starting point.

Store v3 - store synchronisation

• Goal: Upgrade the Store service capability in the network from a collection of local, unsynchronised,
  semi-centralised (trusted) service nodes to a decentralised service capability in the network with inter-node synchronisation.
• Deliverables: (2), (3)
• Research tracks: Message Reliability
• Estimated date of completion: 2024Q2

Building on Store v3-beta, this version of Store includes basic synchronisation between nodes.
This will probably include:

• a protocol/heuristic to resume store services after an offline period
• a protocol/heuristic to periodically compare local key-value store with other nodes and find missing keys
• a protocol/heuristic to periodically download the messages (values) for missing keys from other store nodes

Note that this can either be

• (i) designing a new synchronisation protocol built into the Store protocol
• (ii) adapting existing synchronisation building blocks (e.g. a Prolly Tree library) into Store, or
• (iii) simply swapping the key-value store itself to a synchronised backend, such as GossipLog

IMO (iii) should be pursued as the preferred option, as far as possible.

[SionoiS]

Research: waku-org/research#80
Nwaku:

• feat: Integrate Negentropy RBSR library for Waku Sync nwaku#2426
• feat: reference implementation for Store v3 protocol nwaku#2425

[fryorcraken]

Store v3-beta message hashes

I believe the cursor bug led to some delay in integrating in client side.

What's our estimated completion date for the epics including some dogfooding
@Ivansete-status @weboko @richard-ramos (across all clients?)

Store v3 - store synchronisation

I believe this is deployed, Is it enabled? I am guessing best to have "count store message" to properly dogfood and confirm working?
Any other work missing? How much hardening has happened?
@jm-clius @Ivansete-status

[richard-ramos]

From the point of view of go-waku and status-go nothing pending to be done except merging this PR status-im/status-go#5123, which has been open for a while and rebased from time to time against latest changes from master branch in both go-waku and status-go. @Ivansete-status and I dogfooded this on status.staging last week using this desktop PR status-im/status-desktop#15615

Once shards.test fleet is upgraded to v0.31 the status-go PR can be merged

[fryorcraken]

Delivered