GossipSub v1.2: IDONTWANT control message and priority queue. #553

ppopth · 2024-01-15T18:32:36Z

Sending IDONTWANT

Implement a smart queue
Add priorities to the smart queue
Put IDONTWANT packets into the smart priority queue as soon as the node gets the packets

Handling IDONTWANT

Use a map to remember the message ids whose IDONTWANT packets have been received
Implement max_idontwant_messages (ignore the IDONWANT packets if the max is reached)
Clear the message IDs from the cache after 3 heartbeats
Hash the message IDs before putting them into the cache.

More requested features

Add a feature test to not send IDONTWANT if the other side doesnt support it

Spec: libp2p/specs#548

vyzo

So far looking good overall.

However, I think using a slice to back the queue is problematic at many levels as it might leak space with a growing array, and also hang on to too much space.

I would very much prefer using a dequeue for this.

comm.go

rpc_queue.go

ppopth · 2024-03-02T06:17:03Z

However, I think using a slice to back the queue is problematic at many levels as it might leak space with a growing array, and also hang on to too much space.

The unused space will be freed eventually https://stackoverflow.com/a/26863706

ppopth · 2024-03-05T09:47:36Z

I'm not really sure if we need dfd81e8 or not. I guess adding a non-standard flag to pb/trace.proto would break some users of the tracers?

vyzo · 2024-03-22T08:58:36Z

I'm not really sure if we need dfd81e8 or not. I guess adding a non-standard flag to pb/trace.proto would break some users of the tracers?

Yeah, let's not break compatibility with existing code, we don't need it.

Any progress other than that?

ppopth · 2024-04-08T09:51:36Z

Any progress other than that?

I haven't implemented the second half. Does the first half looks good to you?

vyzo · 2024-04-08T09:58:08Z

yes, looks reasonable so far.

ppopth · 2024-05-01T18:07:49Z

@vyzo I finished everything already. Sorry for the delay. I was in vacation and was a bit busy lately. Thank you for your patience.

vyzo

Generally looks good, but lets make the counter clear less aggressive.

gossipsub.go

ppopth · 2024-05-03T17:25:28Z

The message ids from IDONTWANT can be very large, so hashing it before keeping it in memory quite makes sense. The same thing is already implemented in nim-libp2p vacp2p/nim-libp2p#1090

vyzo · 2024-05-03T17:28:58Z

sure, but it needs to be cleared, otherwise its memroy leak and dos waiting to happen. Or at best the feature breaks itself for long running nodes.

I suggest for each IDW keep a counter of how many heartbeats it survived, and after 3 (to match the IHAVE ads) it should be cleared.

ppopth · 2024-06-11T10:03:31Z

@vyzo It's all done. Sorry for the delay.

vyzo · 2024-06-14T16:31:34Z

ok, can you rebase on master?

chaitanyaprem · 2024-08-06T15:08:33Z

@vyzo wondering if this PR ready to be merged or are you waiting for anything to be fixed/addressed?

This seems to improve bandwidth usage and we want to take advantage of that.

vyzo

I think in general it looks good, but i do want to do another pass.

In the meantime:

can you bump the version to 1.2
can you add a feature test to not send IDONTWANT if the other side doesnt support it?

I will also invoke @Stebalien for review.

chaitanyaprem · 2024-08-06T15:27:57Z

I think in general it looks good, but i do want to do another pass.

In the meantime:
* can you bump the version to 1.2

* can you add a feature test to not send IDONTWANT if the other side doesnt support it?
I will also invoke @Stebalien for review.

Haven't looked at the PR in detail, but will give it a try tomorrow.

rpc_queue.go

comm.go

gossipsub.go

Since we want to implement a priority queue later, we need to replace the normal sending channels with the new smart structures first.

UrgentPush allows you to push an rpc packet to the front of the queue so that it will be popped out fast.

Most importantly, this commit adds a new method called PreValidation to the interface PubSubRouter, which will be called right before validating the gossipsub message. In GossipSubRouter, PreValidation will send the IDONTWANT controll messages to all the mesh peers of the topics of the received messages.

When receiving IDONTWANTs, the host should remember the message ids contained in IDONTWANTs using a hash map. When receiving messages with those ids, it shouldn't forward them to the peers who already sent the IDONTWANTs. When the maximum number of IDONTWANTs is reached for any particular peer, the host should ignore any excessive IDONTWANTs from that peer.

If the messages IDs received from IDONTWANTs are older than 3 heartbeats, they should be removed from the IDONTWANT cache.

Rather than keeping the raw message ids, keep their hashes instead to save memory and protect again memory DoS attacks.

ppopth · 2024-08-15T14:12:53Z

@vyzo I didn't get any error with FuzzAppendOrMergeRPC, did you?

$ go test -v -run FuzzAppendOrMergeRPC
=== RUN   FuzzAppendOrMergeRPC
--- PASS: FuzzAppendOrMergeRPC (0.00s)
PASS
ok  	github.com/libp2p/go-libp2p-pubsub	0.005s

vyzo · 2024-08-15T15:03:38Z

thats a flake then.
Ok, disable and open follow up issue!

ppopth · 2024-08-15T15:38:34Z

done

vyzo · 2024-08-15T16:35:59Z

CI failures - seems like a faulty commit.

vyzo · 2024-08-15T16:38:48Z

@ppopth let's reenable the Fuzz test, it seems it is fine: see #574

gossipsub.go

rpc_queue.go

This reverts commit f141e13.

ppopth · 2024-08-16T06:32:45Z

@Stebalien @vyzo I think it's all done

vyzo · 2024-08-16T15:02:15Z

running the CI, will merge once it greens.

ppopth · 2024-08-16T15:07:28Z

should I squash the fix-up commits?

vyzo

Looks good! Thank you!

vyzo · 2024-08-16T15:08:37Z

should I squash the fix-up commits?

no need, I will squash merge anyway

vyzo self-requested a review January 15, 2024 18:34

vyzo reviewed Feb 10, 2024

View reviewed changes

comm.go Show resolved Hide resolved

rpc_queue.go Outdated Show resolved Hide resolved

rpc_queue.go Outdated Show resolved Hide resolved

vyzo reviewed May 2, 2024

View reviewed changes

gossipsub.go Outdated Show resolved Hide resolved

gossipsub.go Show resolved Hide resolved

ppopth force-pushed the idonthave branch 2 times, most recently from 11fed73 to d44d15a Compare June 20, 2024 05:42

vyzo reviewed Aug 6, 2024

View reviewed changes

vyzo requested a review from Stebalien August 6, 2024 15:13

Stebalien reviewed Aug 6, 2024

View reviewed changes

chaitanyaprem mentioned this pull request Aug 7, 2024

Idonthave chaitanyaprem/go-libp2p-pubsub#1

Open

3 tasks

ppopth added 9 commits August 9, 2024 13:09

Replace sending channel with the smart rpcQueue

6877d7b

Since we want to implement a priority queue later, we need to replace the normal sending channels with the new smart structures first.

Implement UrgentPush in the smart rpcQueue

7ddc486

UrgentPush allows you to push an rpc packet to the front of the queue so that it will be popped out fast.

Add IDONTWANT to rpc.proto and trace.proto

dc45ebb

Test GossipSub IDONWANT sending

22d9773

Send IDONWANT only for large messages

62daad4

Clear expired message IDs from the IDONTWANT cache

4e930a7

If the messages IDs received from IDONTWANTs are older than 3 heartbeats, they should be removed from the IDONTWANT cache.

Keep the hashes of IDONTWANT message ids instead

1afead5

Rather than keeping the raw message ids, keep their hashes instead to save memory and protect again memory DoS attacks.

ppopth added 3 commits August 15, 2024 20:50

Increase GossipSubMaxIDontWantMessages to 1000

e5704b6

fixup! Handle IDONTWANT control messages

f4b05f4

Skip TestGossipsubConnTagMessageDeliveries

e5c971c

Skip FuzzAppendOrMergeRPC

f141e13

ppopth mentioned this pull request Aug 15, 2024

Fix flaky tests TestGossipsubConnTagMessageDeliveries #574

Open

Stebalien reviewed Aug 15, 2024

View reviewed changes

gossipsub.go Show resolved Hide resolved

gossipsub.go Outdated Show resolved Hide resolved

rpc_queue.go Outdated Show resolved Hide resolved

rpc_queue.go Outdated Show resolved Hide resolved

vyzo mentioned this pull request Aug 15, 2024

chore: upgrade go-libp2p #575

Merged

ppopth added 5 commits August 16, 2024 09:25

Revert "Skip FuzzAppendOrMergeRPC"

0d9f553

This reverts commit f141e13.

fixup! Send IDONWANT only for large messages

baba9e1

fixup! fixup! Keep the hashes of IDONTWANT message ids instead

9bdeb9b

fixup! Implement UrgentPush in the smart rpcQueue

6491d8e

fixup! Use AfterFunc instead implementing our own

79dcdc8

Stebalien approved these changes Aug 16, 2024

View reviewed changes

vyzo approved these changes Aug 16, 2024

View reviewed changes

vyzo changed the title ~~Go IDONTWANT~~ GossipSub v1.2: IDONTWANT control message and priority queue. Aug 16, 2024

vyzo merged commit b421b3a into libp2p:master Aug 16, 2024
7 checks passed

vyzo mentioned this pull request Aug 16, 2024

[GossipSub 1.2] IDONTWANT control message libp2p/specs#548

Merged

ppopth deleted the idonthave branch August 19, 2024 03:41

cortze mentioned this pull request Sep 24, 2024

Add trace support for IDONTWANT control messages probe-lab/hermes#37

Merged

nisdas mentioned this pull request Sep 24, 2024

Update Pubsub Library To Use Gossipsub 1.2 prysmaticlabs/prysm#14428

Merged

cortze mentioned this pull request Sep 25, 2024

Possible upgrade to lighthouse's IDONTWANT GossipSub implementation sigp/lighthouse#6437

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GossipSub v1.2: IDONTWANT control message and priority queue. #553

GossipSub v1.2: IDONTWANT control message and priority queue. #553

ppopth commented Jan 15, 2024 •

edited

Loading

vyzo left a comment

ppopth commented Mar 2, 2024

ppopth commented Mar 5, 2024

vyzo commented Mar 22, 2024

ppopth commented Apr 8, 2024

vyzo commented Apr 8, 2024

ppopth commented May 1, 2024

vyzo left a comment

ppopth commented May 3, 2024

vyzo commented May 3, 2024 •

edited

Loading

ppopth commented Jun 11, 2024

vyzo commented Jun 14, 2024

chaitanyaprem commented Aug 6, 2024

vyzo left a comment

chaitanyaprem commented Aug 6, 2024

ppopth commented Aug 15, 2024

vyzo commented Aug 15, 2024

ppopth commented Aug 15, 2024

vyzo commented Aug 15, 2024

vyzo commented Aug 15, 2024

ppopth commented Aug 16, 2024

vyzo commented Aug 16, 2024

ppopth commented Aug 16, 2024

vyzo left a comment

vyzo commented Aug 16, 2024 •

edited

Loading

GossipSub v1.2: IDONTWANT control message and priority queue. #553

GossipSub v1.2: IDONTWANT control message and priority queue. #553

Conversation

ppopth commented Jan 15, 2024 • edited Loading

vyzo left a comment

Choose a reason for hiding this comment

ppopth commented Mar 2, 2024

ppopth commented Mar 5, 2024

vyzo commented Mar 22, 2024

ppopth commented Apr 8, 2024

vyzo commented Apr 8, 2024

ppopth commented May 1, 2024

vyzo left a comment

Choose a reason for hiding this comment

ppopth commented May 3, 2024

vyzo commented May 3, 2024 • edited Loading

ppopth commented Jun 11, 2024

vyzo commented Jun 14, 2024

chaitanyaprem commented Aug 6, 2024

vyzo left a comment

Choose a reason for hiding this comment

chaitanyaprem commented Aug 6, 2024

ppopth commented Aug 15, 2024

vyzo commented Aug 15, 2024

ppopth commented Aug 15, 2024

vyzo commented Aug 15, 2024

vyzo commented Aug 15, 2024

ppopth commented Aug 16, 2024

vyzo commented Aug 16, 2024

ppopth commented Aug 16, 2024

vyzo left a comment

Choose a reason for hiding this comment

vyzo commented Aug 16, 2024 • edited Loading

ppopth commented Jan 15, 2024 •

edited

Loading

vyzo commented May 3, 2024 •

edited

Loading

vyzo commented Aug 16, 2024 •

edited

Loading