feat(nodebuilder/header): Bootstrap from previously seen peers #2507

renaynay · 2023-07-21T11:00:06Z

Provides PIDStore to header module so that it can be used in peerTracker and replaces mem peerstore.Peerstore with on-disk peerstore.Peerstore so that peerTracker can quickly bootstrap itself with previously-seen peers and allow syncer to initialise its sync target from tracked peers rather than trusted so long as it has a subjective head within the trusting period.

Overrides #2133

Closes #1851, mitigates issues resulting from #1623

Swamp integration tests to follow (tracked in #2506)

Future note:

This PR introduces a soon-to-be deprecated feature from libp2p (on-disk peerstore). Once libp2p deprecates and removes this feature, the PIDStore will have to become a PeerAddrStore such that it can save addr info of good peers to disk instead of just their IDs.

renaynay · 2023-08-01T09:23:53Z

This is what it looks like to bootstrap from previously-seen peers:

2023-08-01T11:22:06.626+0200	INFO	module/header	header/config.go:58	No trusted peers in config, initializing with default bootstrappers as trusted peers
2023-08-01T11:22:06.630+0200	INFO	header/p2p	p2p/exchange.go:81	client: starting client	{"protocol ID": "/mocha-3/header-ex/v0.0.3"}
2023-08-01T11:22:06.789+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWDRSJMbH3PS4dRDa11H7Tk615aqTUgkeEKz4pwd4sS6fN"}
2023-08-01T11:22:06.789+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWEk7cxtjQCC7kC84Uhs2j6dAHjdbwYnPcvUAqmj6Zsry2"}
2023-08-01T11:22:06.929+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWBE4QcFXZzENf2VRo6Y5LBvp9gzmpYRHKCvgGzEYj7Hdn"}
2023-08-01T11:22:06.929+0200	INFO	pidstore	pidstore/pidstore.go:58	Loaded peers from disk	{"amount": 101}
2023-08-01T11:22:06.929+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWGQZ5Joykf6YSGXLMVVBtvwrivQydZHgVRsE9PhuHkNuZ"}
2023-08-01T11:22:06.929+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWK6G3P6NhjX2tB37ee3cLXdK31WYNzYgprUS3Qap11idS"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWSDFuWjDUUKFv69BUDXVDtuAZoYeR8ssC4rvfgWt4PdD3"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWSsfZRdQ7EmafNVEe9EN21faZTDyEZ9Rf9GdKQ5WwUuxr"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWLLZTVJ4mMKogSrkhiNs7K2WjVLQYbkF179UeobGh93CN"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWKUn9rkTKmT2DwhpzgGJyMPMWxx1HARxmmGAimzv4Ze1a"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWBE4QcFXZzENf2VRo6Y5LBvp9gzmpYRHKCvgGzEYj7Hdn"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWEk7cxtjQCC7kC84Uhs2j6dAHjdbwYnPcvUAqmj6Zsry2"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWCkp47ByvXHvsX9hm8UTyjzdyUxevVafdLYYbtc1mfVew"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWDRSJMbH3PS4dRDa11H7Tk615aqTUgkeEKz4pwd4sS6fN"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWL8xcw8ouxBhBsZbavG6DDjsy3msF15sL5ZvJ1GgaXpJk"}
2023-08-01T11:22:06.930+0200	INFO	fraudserv	fraudserv/helpers.go:29	joining topic	{"id": "/mocha-3/fraud-sub/badencoding/v0.0.1"}
2023-08-01T11:22:06.930+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWB8Y9og4qzSp3pbdVX2Sp1mvfj38K2rLnnqYYxBmqw4Mq"}
2023-08-01T11:22:06.930+0200	INFO	fraudserv	fraudserv/service.go:100	starting fraud proof service	{"protocol ID": "/mocha-3/fraud/v0.0.1"}
2023-08-01T11:22:06.932+0200	INFO	header/store	store/store.go:176	loaded head	{"height": 186625, "hash": "364F901CA809724712718032D5D307A1D02800F9485A60CE2151D2B35A0F7748"}
2023-08-01T11:22:06.932+0200	INFO	header/p2p	p2p/subscriber.go:40	joining topic	{"topic ID": "/mocha-3/header-sub/v0.0.1"}
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:106	requesting head
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:143	requesting head from tracked peers	{"amount": 4}
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:316	requesting peer	{"peer": "12D3KooWLLZTVJ4mMKogSrkhiNs7K2WjVLQYbkF179UeobGh93CN"}
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:316	requesting peer	{"peer": "12D3KooWDRSJMbH3PS4dRDa11H7Tk615aqTUgkeEKz4pwd4sS6fN"}
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:316	requesting peer	{"peer": "12D3KooWSsfZRdQ7EmafNVEe9EN21faZTDyEZ9Rf9GdKQ5WwUuxr"}
2023-08-01T11:22:06.937+0200	DEBUG	header/p2p	p2p/exchange.go:316	requesting peer	{"peer": "12D3KooWKUn9rkTKmT2DwhpzgGJyMPMWxx1HARxmmGAimzv4Ze1a"}
2023-08-01T11:22:06.971+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWBdbFpqxNFYiHwNDkNZf9pc5BD6gn57v7Qgtxn9RYg7hb"}
2023-08-01T11:22:06.992+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWLb1YvH9VMP91yyUcotATY1E5T4t2bbwA4pcXKVbijLAx"}
2023-08-01T11:22:06.994+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWMLvJpeBWS6tkZsFVBUJ58VuwLy6f78AsVXrzaq1mR6i9"}
2023-08-01T11:22:07.003+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWMwq4dmSZFcBbTzTat6xdcogxenqQ4ejvVwRXLd4L1JWk"}
2023-08-01T11:22:07.004+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWAds1S1MXFLqMo4PqcmSNRbL2RueKhsaXAPEBcTHnmH6Y"}
2023-08-01T11:22:07.004+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWJDQMmcpqq47E4bz76goCzBjVsHCHfcCPfqpXhpRy62ks"}
2023-08-01T11:22:07.004+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWAXrNrs83aJ6HrUE1n8xPRAebC2pJ1dY9KGcVyA5UkvxM"}
2023-08-01T11:22:07.019+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWG9PPLDdRNEwMXRzqstq1URvKbtDBEKHr9mCPnVjkPqtf"}
2023-08-01T11:22:07.019+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWPUhscayL76uwUbjZibasyGHReGvDKXWyXy3ibmuMSW9W"}
2023-08-01T11:22:07.041+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWHmGjbhctWFqpJUWxiwJz2ujXNeE8EEs11dGpbEHVTujc"}
2023-08-01T11:22:07.069+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWJvvACo1drUpckDsnqwLRbU8VufrVRinHUj58VQNA3qPE"}
2023-08-01T11:22:07.076+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWPuVHz6M484B7MuUnQNCN5LeAa5SE2nZcyTPnHkW7XFFK"}
2023-08-01T11:22:07.117+0200	INFO	canonical-log	swarm/swarm_dial.go:493	CANONICAL_PEER_STATUS: peer=12D3KooWJZ1P3tdXYQjPYzsVg6KmTdrPqYRaQpeSdHo3PMiGSyqJ addr=/ip4/89.117.57.27/udp/2121/quic-v1 sample_rate=100 connection_status="established" dir="outbound"
2023-08-01T11:22:07.135+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWRx8TH4Z7wLMc8p6utpyBdxM9eNMsR3puw3VyxEprrCDA"}
2023-08-01T11:22:07.163+0200	DEBUG	header/p2p	p2p/peer_tracker.go:107	connected to peer	{"id": "12D3KooWN3mT9oKMkFKs3XCJTuwmjnUVdcLAXnFUSFDTqzXBrKgp"}
2023-08-01T11:22:07.173+0200	INFO	header/sync	sync/sync_head.go:133	new network head	{"height": 186972, "hash": "D726229089A93E7472C428AE76D689F9C4C5AB746830297C6BA3E4BE8A882937"}

renaynay · 2023-08-11T09:13:08Z

Blocked on release from go-header

codecov-commenter · 2023-09-05T11:55:19Z

Codecov Report

Merging #2507 (4520590) into main (96388ea) will increase coverage by 0.02%.
The diff coverage is 83.33%.

@@            Coverage Diff             @@
##             main    #2507      +/-   ##
==========================================
+ Coverage   51.00%   51.02%   +0.02%     
==========================================
  Files         158      158              
  Lines       10644    10657      +13     
==========================================
+ Hits         5429     5438       +9     
- Misses       4738     4740       +2     
- Partials      477      479       +2

Files Changed	Coverage Δ
libs/pidstore/pidstore.go	`62.50% <75.00%> (+2.50%)`	⬆️
nodebuilder/header/constructors.go	`58.00% <100.00%> (+0.85%)`	⬆️
nodebuilder/header/module.go	`48.52% <100.00%> (+1.55%)`	⬆️
nodebuilder/p2p/misc.go	`65.00% <100.00%> (ø)`

... and 6 files with indirect coverage changes

…k peerstore.Peerstore

Wondertan

help me

The help brigade is OOO

libs/pidstore/pidstore.go

nodebuilder/header/constructors.go

nodebuilder/p2p/misc.go

renaynay added area:p2p kind:feat Attached to feature PRs labels Jul 21, 2023

renaynay self-assigned this Jul 21, 2023

renaynay force-pushed the bootstrap-prev-seen branch from 4e8e827 to 3aa19a9 Compare August 11, 2023 09:08

renaynay force-pushed the bootstrap-prev-seen branch from 3aa19a9 to 2712e73 Compare September 5, 2023 08:54

renaynay marked this pull request as ready for review September 5, 2023 12:58

renaynay requested review from Wondertan, vgonkivs, distractedm1nd and walldiss as code owners September 5, 2023 12:58

renaynay mentioned this pull request Sep 5, 2023

libs/pidstore: Transition to PeerAddrStore once libp2p deprecates on-disk peerstore #2659

Open

renaynay added 4 commits September 5, 2023 15:28

feat(nodebuilder/header): Provide PIDStore to exchange and use on-dis…

2c4b926

…k peerstore.Peerstore

fix(nodebuilder): fix exchange test

29f6628

fix(nodebuilder/header): revert change to exchange constructor

718fcbb

fix(go.mod): put that thing back where it came from or so help me

7e84dc1

renaynay force-pushed the bootstrap-prev-seen branch from 4eb787e to 7e84dc1 Compare September 5, 2023 13:28

Wondertan reviewed Sep 5, 2023

View reviewed changes

libs/pidstore/pidstore.go Show resolved Hide resolved

nodebuilder/header/constructors.go Outdated Show resolved Hide resolved

nodebuilder/header/constructors.go Outdated Show resolved Hide resolved

remove relics

4520590

Wondertan approved these changes Sep 5, 2023

View reviewed changes

vgonkivs reviewed Sep 5, 2023

View reviewed changes

nodebuilder/p2p/misc.go Show resolved Hide resolved

vgonkivs approved these changes Sep 5, 2023

View reviewed changes

renaynay merged commit c146a0a into celestiaorg:main Sep 5, 2023
12 of 14 checks passed

renaynay deleted the bootstrap-prev-seen branch September 5, 2023 14:07

renaynay mentioned this pull request Sep 5, 2023

Feat: Bootstrap from previously seen peers #2133

Closed

5 tasks

faddat mentioned this pull request Feb 22, 2024

feat: go 1.22 and pebble v1.1.0 for celestia-node #3202

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(nodebuilder/header): Bootstrap from previously seen peers #2507

feat(nodebuilder/header): Bootstrap from previously seen peers #2507

renaynay commented Jul 21, 2023 •

edited

Loading

renaynay commented Aug 1, 2023

renaynay commented Aug 11, 2023

codecov-commenter commented Sep 5, 2023 •

edited

Loading

Wondertan left a comment

feat(nodebuilder/header): Bootstrap from previously seen peers #2507

feat(nodebuilder/header): Bootstrap from previously seen peers #2507

Conversation

renaynay commented Jul 21, 2023 • edited Loading

Future note:

renaynay commented Aug 1, 2023

renaynay commented Aug 11, 2023

codecov-commenter commented Sep 5, 2023 • edited Loading

Codecov Report

Wondertan left a comment

Choose a reason for hiding this comment

renaynay commented Jul 21, 2023 •

edited

Loading

codecov-commenter commented Sep 5, 2023 •

edited

Loading