Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor Subnet Search #8048

Merged
merged 24 commits into from
Dec 12, 2020
Merged

Refactor Subnet Search #8048

merged 24 commits into from
Dec 12, 2020

Conversation

nisdas
Copy link
Member

@nisdas nisdas commented Dec 4, 2020

What type of PR is this?

Feature Improvement

What does this PR do? Why is it needed?

  • Previously Subnet search was very slow and did not work for the most part, this PR revamps it and the underlying
    semantics behind it.
  • No more multi-threaded search for separate subnets, instead searches are now sequential. Given that peers subscribe
    to multiple subnets, this is a saner way to approach filtering for more valid peers.
  • Revamp references in attester and aggregator subnet search.

Which issues(s) does this PR fix?

Fixes #

Other notes for review

@nisdas nisdas marked this pull request as ready for review December 6, 2020 02:17
@nisdas nisdas requested a review from a team as a code owner December 6, 2020 02:17
@nisdas nisdas added Ready For Review Networking P2P related items labels Dec 6, 2020
// for a subnet. So that even in the event of poor peer
// connectivity, we can still broadcast an attestation.
func (s *Service) hasPeerWithSubnet(topic string) bool {
return len(s.pubsub.ListPeers(topic+s.Encoding().ProtocolSuffix())) >= 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is the only way you can access this information, but i worry it will be expensive. We might want to add HasPeer(topic string) to libp2p-pubsub. Just a thought.

Copy link
Member Author

@nisdas nisdas Dec 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good point, although the API isn't in our direct control and any proposed changes will take some time to be discussed/implemented upstream

Comment on lines 57 to 58
go func() {
wg.Add(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
go func() {
wg.Add(1)
wg.Add(1)
go func() {

I think you need to add this outside of the anonymous function.

redblaze4209
redblaze4209 previously approved these changes Dec 6, 2020
prestonvanloon
prestonvanloon previously approved these changes Dec 7, 2020

// filterNodes wraps an iterator such that Next only returns nodes for which
// the 'check' function returns true. This custom implementation also
// checks for context deadlines so that t
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

godoc incomplete. It would be nice to have an explanation for why we are writing a custom iterator in Prysm

@@ -18,10 +17,14 @@ var attestationSubnetCount = params.BeaconNetworkConfig().AttestationSubnetCount

var attSubnetEnrKey = params.BeaconNetworkConfig().AttSubnetKey

const peersRequiredInSubnetSearch = 20
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be a config parameter?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

beacon-chain/p2p/iterator.go Show resolved Hide resolved
@@ -18,10 +17,14 @@ var attestationSubnetCount = params.BeaconNetworkConfig().AttestationSubnetCount

var attSubnetEnrKey = params.BeaconNetworkConfig().AttSubnetKey

const peersRequiredInSubnetSearch = 20
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@prestonvanloon prestonvanloon merged commit 29804fa into develop Dec 12, 2020
@delete-merged-branch delete-merged-branch bot deleted the revampSubnetSearch branch December 12, 2020 02:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Networking P2P related items
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants