Seeker autoconnect #7798
Open
Seeker autoconnect #7798
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Based on gossip sync data from random network peers, listening to only 5 peers will not reliably catch all gossip traffic. For now, add extra redundancy.
endothermicdev
force-pushed
the
seeker-autoconnect
branch
from
72101ef
to
d4133eb
Compare
This can help us backfill any missing gossip if our current peers haven't been the most reliable. Changelog-Changed: Gossipd requests a full sync from a random peer every hour.
This will be used to drop underperforming gossipers instead of choosing at random.
Previously they were chosen at random. We should instead prefer to strop streaming from the peer who has provided the least novel gossip.
This will allow all gossipers to be rotated in ~5 hours similar to how it operated with half as many gossip streaming peers.
endothermicdev
force-pushed
the
seeker-autoconnect
branch
from
d4133eb
to
caca9aa
Compare
in test_gossip_force_broadcast_channel_msgs now that the seeker is asking for periodic full gossip syncs.
Gossipd uses this to ask lightningd -> connectd to initiate a connection to a new gossip peer. This can be used when there are insufficient peers already connected to gossip with. Changelog-Changed: Gossipd can now request connections to additional nodes for improved gossip sync
This does not validate a node announcement and address, but it does select a node at random from the gossmap and asks lightningd to attempt a connection to it.
endothermicdev
force-pushed
the
seeker-autoconnect
branch
from
caca9aa
to
5c15ef8
Compare
endothermicdev
force-pushed
the
seeker-autoconnect
branch
from
5c15ef8
to
bb81f52
Compare
rustyrussell
requested changes
Nov 15, 2024
Comment on lines
+974
to
+977
| static struct node_and_addrs *get_random_node(const tal_t *ctx, | ||
| struct seeker *seeker) | ||
| { | ||
| struct gossmap *gossmap = gossmap_manage_get_gossmap(seeker->daemon->gm); |
There was a problem hiding this comment.
This is better done by providing a "struct gossmap_node *gossmap_random_node(const struct gossmap *map);" in gossmap.c, which can use nodeidx_htable_pick. Then iterate until NULL, or you find one with an address.
Comment on lines
+1036
to
+1042
| if(!random_node->addrs || tal_count(random_node->addrs) == 0) | ||
| status_broken("seeker: random gossip node missing address"); | ||
|
|
||
| u8 *msg = towire_gossipd_connect_to_peer(NULL, random_node->id, | ||
| random_node->addrs); | ||
| daemon_conn_send(seeker->daemon->master, take(msg)); | ||
| tal_free(random_node); |
There was a problem hiding this comment.
Simplify: don't hand addrs here, since connectd already has that logic internally.
| /* FIXME: store random sync peer and don't select next time. */ | ||
| status_peer_debug(&random_peer->id, | ||
| "seeker: chosen for periodic full sync"); | ||
| normal_gossip_start(seeker,random_peer, true); |
There was a problem hiding this comment.
This doesn't do a sync, it just starts gossip from this point.
You need to send query_channel_range, then query_short_channel_ids for any we didn't know about in the reply.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Important
24.11 FREEZE NOVEMBER 7TH: Non-bugfix PRs not ready by this date will wait for 25.02.
Checklist
This builds on #7768 to make our gossip seeker more capable at syncing the network graph. In the case of nodes with a small number of peers, it may be impossible to get a reasonably complete enough gossip sync. In this case, choosing a node at random to try to connect and request additional gossip seems reasonable.
Testing from 14 Sept 2024 found that only 30% of 60 nodes tested were able to sync >90% of the current network graph. This should significantly aid small nodes in syncing their gossip.
Changelog-Changed: Gossipd can now request connections to additional nodes for improved gossip sync