-
Notifications
You must be signed in to change notification settings - Fork 448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: auto dial discovered peers #349
Conversation
That's interesting - can you foresee any strategies other than low and all? |
After thinking through this a bit more I put together some notes. The main take away would be to change the behavior of libp2p to:
ScenariosIn any scenario, if a peer is discovered it should be added to the PeerBook. This ensures that 1. Joining the networkThe node is new and needs to join the network. It currently has 0 peers. Action to takeConnect to discovered peers. This should have some degree of concurrency limiting. While the case should be low, if we immediately discover more peers than our high watermark we should avoid dialing them all. 2. Connected to someThe node is connected to other nodes. The current number of connections is less than the desired low watermark. Action to takeConnect to discovered peers. This should have some degree of concurrency limiting. The concurrency may need to be modified to reflect the current number of peers connected. The more peers we have, the lower the concurrency may need to be. 3. Connected to enoughDiscovery Mechanisms: Ambient Discovery and Active Discovery Action to takeNone. If we are connected to enough peers, the low watermark, we should not connect to discovered peers. As other peers discover us, they may connect to us based on their current scenario. For example, a long running node with adequate peers is on an MDNS network. A new peer joins the network and both become aware of each other. The new peer should be the peer that dials, as it has too few peers. The existing node has no reason to dial the new peer, but should keep a record of it in case it later becomes an important node due to its contents/capabilities. Avoiding dials above the low watermark also allows for a pool of connections to be reserved for application specific actions, such as connecting to a specific content provider via a DHT query to find that content (ipfs-bitswap). 4. Connected to too manyThe node has more connections than it wants. The current number of connections is greater than the high watermark. WIP Connection Manager v2 spec Discovery MechanismsMeans of which a libp2p node discovers other peers. Active DiscoveryThrough active use of the libp2p network, a node may discovery peers.
Ambient DiscoveryLeveraging known addresses, or network discovery mechanisms, a node may discover peers outside of the bounds of the libp2p network.
Thoughts
|
892c1d3
to
2f10234
Compare
This sounds like a good approach 👍 A few questions:
|
There may be value in doing so in the future. I think the first step here will be improving how inbound connections are being handled in the switch and adding the appropriate tags to the connections, which is something the connection manager needs.
No, we shouldn't need the discovery strategies anymore. I pushed up the latest code I have (tests need to be fixed). This changes things over to a configurable
Yes, I think they should. Doing auto dialing makes this a lot easier to know where things are coming from so deprioritizing these calls becomes a lot easier. |
Thanks for clearly exposing this problem and your ideas in this regard @jacobheun ! 🙌 Totally agree with the following reasoning 👍 :
|
👍 thanks for putting so much thought into this @jacobheun - this sounds like a good way forward to limit the number of dials we're doing. |
Can we add your documentation here to the README? |
Since there's a good amount of info there I created a separate readme with the peer discovery / auto dial information. I also updated some language in the main readme and linked to the new readme. |
|
||
if (!this.isStarted()) return | ||
|
||
this.emit('peer:discovery', peerInfo) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why no discovery event if not started?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It keeps a record of those peers and will emit them on startup. This avoids the scenario where you disable autoDial and get a bootstrap or mdns peer discovered result before libp2p starts. If you try to dial and libp2p hasn't finished starting it will fail. This just makes libp2p take care of the logic internally.
src/index.js
Outdated
this.emit('peer:discovery', peerInfo) | ||
this._maybeConnect(peerInfo) | ||
}) | ||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need to unregister this on stop?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This should really run anytime the node starts. It will get run multiple times if the node is started again, though. I can just register this in the constructor to avoid the need to remove and re-add it on a restart.
PEER_DISCOVERY.md
Outdated
* All peers discovered are emitted via `peer:discovery` so applications can take any desired action. | ||
* Libp2p defaults to automatically connecting to new peers, when under the [ConnectionManager](https://github.com/libp2p/js-libp2p-connection-manager) low watermark (minimum peers). | ||
* Applications can disable this via the `peerDiscovery.autoDial` config property, and handle connections themselves. | ||
* Applications who have not disabled this should **never** connect on peer discovery, unless performing a specific action. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe elaborate on what "performing a specific action" is. This is not just "dial with a protocol", you can cold call but it should be in response to some user action.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thinking through this more, I don't think there's any reason for a connect to ever happen on peer discovery with auto dial on. peer:connect
would be a more appropriate event to take action on.
Co-Authored-By: jacobheun <jacobheun@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A couple of nits, otherwise LGTM 👍
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code LGTM, I would like to see how this performs (just haven't had time this week), but we can always iterate if you want to get this merged.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
This PR leverages the strategies detailed at #349 (comment) to implement auto dial in libp2p. This should improve our ability to maintain an appropriate level of connections based on the min and max peers configured for the
ConnectionManager
.Applications leveraging libp2p should ensure they are not dialing discovered peers when using this update. Applications that do wish to handle all dialing can disable
autoDial
in the config.Depends on libp2p/js-libp2p-switch#325