Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(bootstrap): save connected peers as backup bootstrap peers #8856

Merged
merged 4 commits into from
May 25, 2023

Conversation

schomatis
Copy link
Contributor

@schomatis schomatis commented Apr 7, 2022

This PR aims to close #3926.

Every 1 hour (DefaultBootstrapConfig.BackupBootstrapInterval, can override via Internal.BackupBootstrapInterval) we save a number of peers we're connected to in Kubo's datastore. If we can't connect to our "official" bootstrap peers defined in the Bootstrap config (if we don't hit the target in DefaultBootstrapConfig.MinPeerThreshold, again hardcoded, why is this not in the config file?) we resort to connecting to some of the peers from the above list.

How To Manually Test

export IPFS_LOGGING="bootstrap=debug"
ipfs daemon

Running a few seconds we'll see the log:

DEBUG	bootstrap	bootstrap/bootstrap.go:210	saved 8 connected peers (of 8 target) as bootstrap backup in the config

Stop the daemon.

Backup and remove the official bootstrap peers, CAREFUL:

ipfs config --json Bootstrap | tee bootstrap-peers.back
# List of hardcoded bootstrap peers, like: "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN"

ipfs config --json Bootstrap []

At this point, the version in master should not be able to connect to the network:

ipfs daemon
# WARN	bootstrap	bootstrap/bootstrap.go:103	no bootstrap nodes configured: go-ipfs may have difficulty connecting to the network

ipfs swarm peers
# nothing

With this PR:

ipfs daemon
# DEBUG	bootstrap	bootstrap/bootstrap.go:242	not enough bootstrap peers to fill the remaining target of 4 connections, trying backup list

ipfs swarm peers
# normal amount of peers, connected to the network

Restore bootstrap with

ipfs config --json Bootstrap "$(cat bootstrap-peers.back)"

@schomatis schomatis linked an issue Apr 7, 2022 that may be closed by this pull request
@schomatis schomatis requested a review from lidel April 7, 2022 20:28
@schomatis schomatis self-assigned this Apr 7, 2022
config/config.go Outdated Show resolved Hide resolved
@BigLep BigLep added this to the Best Effort Track milestone Apr 22, 2022
@BigLep

This comment was marked as resolved.

@guseggert
Copy link
Contributor

@schomatis: let's put this in the repo datastore for now, so that we are free to change/remove it later without breaking folks.

@schomatis schomatis force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from b4ad48f to efb9fe4 Compare May 5, 2022 14:31
@schomatis schomatis requested a review from guseggert May 5, 2022 14:31
@schomatis
Copy link
Contributor Author

@guseggert Moved to datastore. Ready for review. Will need help with tests if required to land this.

@schomatis schomatis force-pushed the schomatis/feat/bootstrap/save-connected-peers branch 2 times, most recently from 4919894 to f2642f0 Compare May 5, 2022 14:36
@BigLep
Copy link
Contributor

BigLep commented Jun 1, 2022

@guseggert : this is a mental note that I'd like to flag this one for code review Friday so we get it over the line.

@Winterhuman
Copy link
Contributor

Winterhuman commented Oct 5, 2022

@BigLep Any update on this? I've read into the issue and it seems the end goal is to move this functionality into go-libp2p (libp2p/go-libp2p-kad-dht#254), is this still the plan?

@guillaumemichel
Copy link

This feature would be great to have. It enables:

  • less load on bootstrap nodes
  • faster bootstrap (finding self from the known closest peer still online)
  • more balanced RT buckets. Currently empty buckets are filled by querying a random key belonging to this bucket. Hence all the peers inside this bucket will all be very close from this random key, and will not be distributed uniformly in the bucket keyspace, which is desired as it reduces the number of hop during the lookup process.
  • less dependence on central bootstrap nodes, more dependence on known peers. Makes the network more decentralized

@BigLep
Copy link
Contributor

BigLep commented Feb 16, 2023

Thanks for the flags. This has gotten lost amongst all the other priorities that have emerged so far in 2023. I have in the list for 0.20. A key thing we're missing is what is remaining to land this.

@hacdias hacdias requested review from hacdias and removed request for guseggert May 16, 2023 13:50
@hacdias hacdias self-assigned this May 16, 2023
@hacdias hacdias force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from f2642f0 to 94b3b8d Compare May 16, 2023 14:09
@hacdias hacdias requested a review from a team as a code owner May 16, 2023 14:09
@hacdias
Copy link
Member

hacdias commented May 16, 2023

I just rebased this and I will try taking a look at it during this week or the next week. This would be a great addition.

@Jorropo it would be great if you could take a look since you've reviewed it before.

@hacdias hacdias requested a review from Jorropo May 16, 2023 14:14
@hacdias hacdias changed the title feat(bootstrap): save connected peers as backup temporary bootstrap ones feat(bootstrap): save connected peers as backup bootstrap peers May 17, 2023
Copy link
Member

@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went over the code, cleaned up some comments and pushed a fix. Also updated the main PR description with how to test manually. The code looks good to me and works well for me. What I think is left to review:

  • Decide if new variable names are fine, as well as defaults. I have no problems with them.
  • I would like to test this, but I don't see how we'd do it.

cc @ipfs/kubo-maintainers

@hacdias hacdias force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from 8d4f6b3 to 5cd0c4d Compare May 17, 2023 09:42
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits below, but it does not have to be perfect, anything here is better than nothing.
Important part here is to have regression tests that ensure backup bootstrappers are used.

@hacdias I think testing would live in https://github.com/ipfs/kubo/tree/master/test/cli and be similar to swarm_test.go or dht_legacy_test.go:

  1. configure node to have ipfs config --json Bootstrap [] and Discovery.MDNS.Enabled: false (we want it to boot and have no peers)
  2. set bootstrap backup interval to to something small, like 250ms (could be config proposed below inline, or could be env override via os.LookupEnv)
  3. start the daemon, confirm it has no peers (expected)
  4. start the second peer (with default config and Discovery.MDNS.Enabled: false, we want it to be a member of public swarm, but not connect to the first one) and confirm it has some peers
  5. manually connect first one to the second one
  6. confirm the first one learned about other peers (ipfs swarm peers longer than 1)
  7. shut down both daemons, and then start first one again
  8. it should be connected to something thanks to the backed up peers (ipfs swarm peers longer than 1)

(to make this test work offline we could introduce a third node )

core/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
core/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
core/bootstrap/bootstrap.go Outdated Show resolved Hide resolved
@hacdias hacdias force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from a64a86a to 416c6b6 Compare May 17, 2023 12:18
@hacdias
Copy link
Member

hacdias commented May 17, 2023

@lidel I added a test that runs completely separately from the rest of the network. I made a slight change compared to what you suggested. We have 0, 1 and 2. They all have the same config (no bootstrap nodes, no MDNS enabled, 250ms for backup bootstrap interval). The following happens:

  • Start all daemons.
  • 0 and 1 connect to each other. Wait for 0 to save 1 as backup bootstrap.
  • Ensure 0 and 1 are connected to 1 peer (each other). 2 is connected to none.
  • Stop all daemons.
  • Start 1 and 2. Connect both and ensure they're not connected.
  • Connect 1 and 2 and ensure they are connected.
  • Start 0. Ensure it connects automatically to 1 and discovers 2 via 1. They're all connected.

@hacdias hacdias requested a review from lidel May 17, 2023 12:24
Copy link
Member

@hacdias hacdias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm approving this. I will wait for one more approval since I wrote the test 😃

@hacdias hacdias force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from 416c6b6 to 93aa0f8 Compare May 24, 2023 13:07
@lidel lidel assigned lidel and unassigned schomatis May 25, 2023
@lidel lidel force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from bbed7d9 to 1f4cbd6 Compare May 25, 2023 12:29
@lidel lidel force-pushed the schomatis/feat/bootstrap/save-connected-peers branch from 1f4cbd6 to 7339162 Compare May 25, 2023 12:30
Comment on lines +68 to +72
MinPeerThreshold: 4,
Period: 30 * time.Second,
ConnectionTimeout: (30 * time.Second) / 3, // Perod / 3
BackupBootstrapInterval: 1 * time.Hour,
MaxBackupBootstrapSize: 20,
Copy link
Member

@lidel lidel May 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ℹ️ We can expose these later under Internal.* if needed (in future PRs).

For now, only Internal.BackupBootstrapInterval is exposed for use in the test/cli/backup_bootstrap_test.go test. Not documenting it, as we may change the name/location later.

Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one baked for long enough -- does the basic thing, we can refine in follow-up PRs, but is already way better than nothing.

Thank you all for pushing on this. Added changelog entry, merging when CI passes.

@lidel lidel merged commit 63561f3 into master May 25, 2023
@lidel lidel deleted the schomatis/feat/bootstrap/save-connected-peers branch May 25, 2023 12:39
@BigLep
Copy link
Contributor

BigLep commented May 25, 2023

@lidel : Awesome! I wanted to confirm that the default bootstrap peers will still stay in the list, but not now the list is augmented with additional peers based on past discovery.

The reason I'm asking is because of probelab's measurements that try and ascertain the DHT network node size based on the number of peerids that connect to it.

@hacdias
Copy link
Member

hacdias commented May 25, 2023

@BigLep the default bootstrap peers stays. We add additional backup peers based on our connections. Those will be dialed if the default bootstrap list does not work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Saving previously seen nodes for later bootstrapping
8 participants