Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return peer count #3860

Merged
merged 4 commits into from
Mar 24, 2022
Merged

Return peer count #3860

merged 4 commits into from
Mar 24, 2022

Conversation

dadepo
Copy link
Contributor

@dadepo dadepo commented Mar 17, 2022

Motivation

Implements the endpoint to get the peer count which was not implemented before

Closes #3857

Steps to test or reproduce

curl localhost:9596/eth/v1/node/peer_count should not return just 0 for all values

@codecov
Copy link

codecov bot commented Mar 17, 2022

Codecov Report

Merging #3860 (42aeb9f) into master (94c8e12) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3860   +/-   ##
=======================================
  Coverage   36.11%   36.11%           
=======================================
  Files         325      325           
  Lines        9043     9043           
  Branches     1419     1419           
=======================================
  Hits         3266     3266           
  Misses       5634     5634           
  Partials      143      143           

@dadepo
Copy link
Contributor Author

dadepo commented Mar 17, 2022

Trying the same endpoint on lighthouse after running for about 5 minutes returns count for both connected and disconnected, while for lodestar, the endpoint only returns count for connected and nothing for disconnected or the other status. Still need to see if the current way is the most accurate way of getting the connection info from libp2p.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 17, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: fe831b9 Previous: d8ec024 Ratio
BeaconState.hashTreeRoot - No change 667.00 ns/op 567.00 ns/op 1.18
BeaconState.hashTreeRoot - 1 full validator 149.63 us/op 125.25 us/op 1.19
BeaconState.hashTreeRoot - 32 full validator 2.3100 ms/op 1.9270 ms/op 1.20
BeaconState.hashTreeRoot - 512 full validator 31.524 ms/op 25.680 ms/op 1.23
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 149.88 us/op 126.21 us/op 1.19
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 2.8323 ms/op 2.2782 ms/op 1.24
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 33.212 ms/op 27.811 ms/op 1.19
BeaconState.hashTreeRoot - 1 balances 108.69 us/op 89.489 us/op 1.21
BeaconState.hashTreeRoot - 32 balances 913.27 us/op 765.86 us/op 1.19
BeaconState.hashTreeRoot - 512 balances 9.2639 ms/op 7.3286 ms/op 1.26
BeaconState.hashTreeRoot - 250000 balances 159.66 ms/op 135.97 ms/op 1.17
processSlot - 1 slots 55.214 us/op 55.111 us/op 1.00
processSlot - 32 slots 3.4079 ms/op 2.8736 ms/op 1.19
getCommitteeAssignments - req 1 vs - 250000 vc 6.3838 ms/op 5.2263 ms/op 1.22
getCommitteeAssignments - req 100 vs - 250000 vc 8.9687 ms/op 7.2367 ms/op 1.24
getCommitteeAssignments - req 1000 vs - 250000 vc 9.7164 ms/op 7.7795 ms/op 1.25
computeProposers - vc 250000 25.165 ms/op 20.688 ms/op 1.22
computeEpochShuffling - vc 250000 223.76 ms/op 185.50 ms/op 1.21
getNextSyncCommittee - vc 250000 412.61 ms/op 340.09 ms/op 1.21
altair processAttestation - 250000 vs - 7PWei normalcase 47.685 ms/op 36.204 ms/op 1.32
altair processAttestation - 250000 vs - 7PWei worstcase 42.317 ms/op 36.257 ms/op 1.17
altair processAttestation - setStatus - 1/6 committees join 14.321 ms/op 13.010 ms/op 1.10
altair processAttestation - setStatus - 1/3 committees join 28.356 ms/op 26.485 ms/op 1.07
altair processAttestation - setStatus - 1/2 committees join 42.769 ms/op 36.307 ms/op 1.18
altair processAttestation - setStatus - 2/3 committees join 56.986 ms/op 46.703 ms/op 1.22
altair processAttestation - setStatus - 4/5 committees join 69.667 ms/op 64.630 ms/op 1.08
altair processAttestation - setStatus - 100% committees join 90.395 ms/op 67.005 ms/op 1.35
altair processAttestation - updateEpochParticipants - 1/6 committees join 15.859 ms/op 10.951 ms/op 1.45
altair processAttestation - updateEpochParticipants - 1/3 committees join 31.081 ms/op 22.460 ms/op 1.38
altair processAttestation - updateEpochParticipants - 1/2 committees join 23.815 ms/op 20.767 ms/op 1.15
altair processAttestation - updateEpochParticipants - 2/3 committees join 22.970 ms/op 22.239 ms/op 1.03
altair processAttestation - updateEpochParticipants - 4/5 committees join 26.740 ms/op 23.383 ms/op 1.14
altair processAttestation - updateEpochParticipants - 100% committees join 27.704 ms/op 27.273 ms/op 1.02
altair processAttestation - updateAllStatus 21.996 ms/op 19.078 ms/op 1.15
altair processBlock - 250000 vs - 7PWei normalcase 39.871 ms/op 33.210 ms/op 1.20
altair processBlock - 250000 vs - 7PWei worstcase 121.19 ms/op 103.12 ms/op 1.18
altair processEpoch - mainnet_e81889 921.75 ms/op 788.19 ms/op 1.17
mainnet_e81889 - altair beforeProcessEpoch 385.16 ms/op 314.92 ms/op 1.22
mainnet_e81889 - altair processJustificationAndFinalization 69.843 us/op 65.865 us/op 1.06
mainnet_e81889 - altair processInactivityUpdates 20.468 ms/op 18.253 ms/op 1.12
mainnet_e81889 - altair processRewardsAndPenalties 108.44 ms/op 97.669 ms/op 1.11
mainnet_e81889 - altair processRegistryUpdates 10.532 us/op 7.9720 us/op 1.32
mainnet_e81889 - altair processSlashings 3.4440 us/op 1.3370 us/op 2.58
mainnet_e81889 - altair processEth1DataReset 3.1440 us/op 1.5550 us/op 2.02
mainnet_e81889 - altair processEffectiveBalanceUpdates 7.5827 ms/op 6.9512 ms/op 1.09
mainnet_e81889 - altair processSlashingsReset 17.780 us/op 12.686 us/op 1.40
mainnet_e81889 - altair processRandaoMixesReset 20.197 us/op 17.153 us/op 1.18
mainnet_e81889 - altair processHistoricalRootsUpdate 4.0750 us/op 1.9410 us/op 2.10
mainnet_e81889 - altair processParticipationFlagUpdates 78.208 ms/op 68.277 ms/op 1.15
mainnet_e81889 - altair processSyncCommitteeUpdates 3.0100 us/op 1.4580 us/op 2.06
mainnet_e81889 - altair afterProcessEpoch 264.51 ms/op 222.41 ms/op 1.19
altair processInactivityUpdates - 250000 normalcase 84.670 ms/op 70.438 ms/op 1.20
altair processInactivityUpdates - 250000 worstcase 88.155 ms/op 75.841 ms/op 1.16
altair processParticipationFlagUpdates - 250000 anycase 74.790 ms/op 69.706 ms/op 1.07
altair processRewardsAndPenalties - 250000 normalcase 97.734 ms/op 82.491 ms/op 1.18
altair processRewardsAndPenalties - 250000 worstcase 112.89 ms/op 108.26 ms/op 1.04
altair processSyncCommitteeUpdates - 250000 422.61 ms/op 354.45 ms/op 1.19
Tree 40 250000 create 790.91 ms/op 644.13 ms/op 1.23
Tree 40 250000 get(125000) 395.37 ns/op 325.94 ns/op 1.21
Tree 40 250000 set(125000) 2.3730 us/op 2.0970 us/op 1.13
Tree 40 250000 toArray() 46.680 ms/op 40.997 ms/op 1.14
Tree 40 250000 iterate all - toArray() + loop 47.331 ms/op 42.662 ms/op 1.11
Tree 40 250000 iterate all - get(i) 141.90 ms/op 122.81 ms/op 1.16
MutableVector 250000 create 22.845 ms/op 20.209 ms/op 1.13
MutableVector 250000 get(125000) 14.907 ns/op 12.966 ns/op 1.15
MutableVector 250000 set(125000) 665.78 ns/op 544.34 ns/op 1.22
MutableVector 250000 toArray() 11.635 ms/op 8.5468 ms/op 1.36
MutableVector 250000 iterate all - toArray() + loop 10.314 ms/op 8.7689 ms/op 1.18
MutableVector 250000 iterate all - get(i) 4.5262 ms/op 3.3947 ms/op 1.33
Array 250000 create 6.2417 ms/op 5.6533 ms/op 1.10
Array 250000 clone - spread 2.1882 ms/op 2.3347 ms/op 0.94
Array 250000 get(125000) 1.1260 ns/op 1.1290 ns/op 1.00
Array 250000 set(125000) 1.1040 ns/op 1.1210 ns/op 0.98
Array 250000 iterate all - loop 201.20 us/op 167.82 us/op 1.20
effectiveBalanceIncrements clone Uint8Array 300000 87.651 us/op 77.311 us/op 1.13
effectiveBalanceIncrements clone MutableVector 300000 624.00 ns/op 506.00 ns/op 1.23
effectiveBalanceIncrements rw all Uint8Array 300000 362.24 us/op 301.83 us/op 1.20
effectiveBalanceIncrements rw all MutableVector 300000 193.63 ms/op 170.54 ms/op 1.14
aggregationBits - 2048 els - readonlyValues 212.03 us/op 180.82 us/op 1.17
aggregationBits - 2048 els - zipIndexesInBitList 37.932 us/op 34.468 us/op 1.10
regular array get 100000 times 80.948 us/op 67.431 us/op 1.20
wrappedArray get 100000 times 81.004 us/op 67.432 us/op 1.20
arrayWithProxy get 100000 times 42.342 ms/op 29.929 ms/op 1.41
ssz.Root.equals 1.3150 us/op 1.0990 us/op 1.20
ssz.Root.equals with valueOf() 1.5580 us/op 1.3090 us/op 1.19
byteArrayEquals with valueOf() 1.5110 us/op 1.3000 us/op 1.16
phase0 processBlock - 250000 vs - 7PWei normalcase 9.1972 ms/op 7.9290 ms/op 1.16
phase0 processBlock - 250000 vs - 7PWei worstcase 86.966 ms/op 73.546 ms/op 1.18
phase0 afterProcessEpoch - 250000 vs - 7PWei 247.08 ms/op 202.42 ms/op 1.22
phase0 beforeProcessEpoch - 250000 vs - 7PWei 635.96 ms/op 554.75 ms/op 1.15
phase0 processEpoch - mainnet_e58758 886.35 ms/op 769.03 ms/op 1.15
mainnet_e58758 - phase0 beforeProcessEpoch 493.84 ms/op 437.59 ms/op 1.13
mainnet_e58758 - phase0 processJustificationAndFinalization 72.872 us/op 55.579 us/op 1.31
mainnet_e58758 - phase0 processRewardsAndPenalties 107.74 ms/op 82.775 ms/op 1.30
mainnet_e58758 - phase0 processRegistryUpdates 52.666 us/op 37.075 us/op 1.42
mainnet_e58758 - phase0 processSlashings 3.3660 us/op 1.3820 us/op 2.44
mainnet_e58758 - phase0 processEth1DataReset 3.1790 us/op 1.2640 us/op 2.52
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 6.1859 ms/op 5.7300 ms/op 1.08
mainnet_e58758 - phase0 processSlashingsReset 15.329 us/op 7.2590 us/op 2.11
mainnet_e58758 - phase0 processRandaoMixesReset 16.480 us/op 14.712 us/op 1.12
mainnet_e58758 - phase0 processHistoricalRootsUpdate 3.5580 us/op 1.9510 us/op 1.82
mainnet_e58758 - phase0 processParticipationRecordUpdates 15.372 us/op 12.925 us/op 1.19
mainnet_e58758 - phase0 afterProcessEpoch 212.72 ms/op 182.02 ms/op 1.17
phase0 processEffectiveBalanceUpdates - 250000 normalcase 7.2373 ms/op 6.5273 ms/op 1.11
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 7.6862 ms/op 7.0997 ms/op 1.08
phase0 processRegistryUpdates - 250000 normalcase 44.004 us/op 54.904 us/op 0.80
phase0 processRegistryUpdates - 250000 badcase_full_deposits 3.5374 ms/op 3.1470 ms/op 1.12
phase0 processRegistryUpdates - 250000 worstcase 0.5 2.0829 s/op 1.5127 s/op 1.38
phase0 getAttestationDeltas - 250000 normalcase 15.706 ms/op 12.959 ms/op 1.21
phase0 getAttestationDeltas - 250000 worstcase 16.281 ms/op 12.941 ms/op 1.26
phase0 processSlashings - 250000 worstcase 48.012 ms/op 35.685 ms/op 1.35
shuffle list - 16384 els 15.351 ms/op 12.790 ms/op 1.20
shuffle list - 250000 els 221.54 ms/op 182.94 ms/op 1.21
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 590.27 us/op 461.88 us/op 1.28
pass gossip attestations to forkchoice per slot 18.530 ms/op 15.029 ms/op 1.23
computeDeltas 3.7505 ms/op 3.7272 ms/op 1.01
computeProposerBoostScoreFromBalances 604.01 us/op 503.08 us/op 1.20
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.5902 ms/op 1.7986 ms/op 1.44
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 1.1597 ms/op 707.85 us/op 1.64
BLS verify - blst-native 2.2290 ms/op 1.8562 ms/op 1.20
BLS verifyMultipleSignatures 3 - blst-native 4.5618 ms/op 3.8020 ms/op 1.20
BLS verifyMultipleSignatures 8 - blst-native 9.8410 ms/op 8.1830 ms/op 1.20
BLS verifyMultipleSignatures 32 - blst-native 35.620 ms/op 29.675 ms/op 1.20
BLS aggregatePubkeys 32 - blst-native 48.219 us/op 40.431 us/op 1.19
BLS aggregatePubkeys 128 - blst-native 185.29 us/op 154.30 us/op 1.20
getAttestationsForBlock 85.326 ms/op 59.672 ms/op 1.43
CheckpointStateCache - add get delete 23.764 us/op 17.774 us/op 1.34
validate gossip signedAggregateAndProof - struct 5.5024 ms/op 4.4175 ms/op 1.25
validate gossip signedAggregateAndProof - treeBacked 5.5397 ms/op 4.3703 ms/op 1.27
validate gossip attestation - struct 2.6032 ms/op 2.0805 ms/op 1.25
validate gossip attestation - treeBacked 2.4980 ms/op 2.1108 ms/op 1.18
pickEth1Vote - no votes 12.856 ms/op 10.011 ms/op 1.28
pickEth1Vote - max votes 62.126 ms/op 53.318 ms/op 1.17
pickEth1Vote - Eth1Data hashTreeRoot value x2048 30.121 ms/op 24.974 ms/op 1.21
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 11.552 ms/op 9.7305 ms/op 1.19
pickEth1Vote - Eth1Data fastSerialize value x2048 6.9673 ms/op 5.1990 ms/op 1.34
pickEth1Vote - Eth1Data fastSerialize tree x2048 29.307 ms/op 24.044 ms/op 1.22
bytes32 toHexString 2.2650 us/op 1.6760 us/op 1.35
bytes32 Buffer.toString(hex) 858.00 ns/op 701.00 ns/op 1.22
bytes32 Buffer.toString(hex) from Uint8Array 1.1850 us/op 965.00 ns/op 1.23
bytes32 Buffer.toString(hex) + 0x 855.00 ns/op 710.00 ns/op 1.20
Object access 1 prop 0.42400 ns/op 0.32600 ns/op 1.30
Map access 1 prop 0.36400 ns/op 0.30800 ns/op 1.18
Object get x1000 21.617 ns/op 17.337 ns/op 1.25
Map get x1000 1.1770 ns/op 0.98600 ns/op 1.19
Object set x1000 134.00 ns/op 103.05 ns/op 1.30
Map set x1000 83.461 ns/op 64.932 ns/op 1.29
Return object 10000 times 0.44730 ns/op 0.37470 ns/op 1.19
Throw Error 10000 times 6.9766 us/op 5.7847 us/op 1.21
enrSubnets - fastDeserialize 64 bits 1.5880 us/op 1.2830 us/op 1.24
enrSubnets - ssz BitVector 64 bits 19.876 us/op 16.760 us/op 1.19
enrSubnets - fastDeserialize 4 bits 572.00 ns/op 480.00 ns/op 1.19
enrSubnets - ssz BitVector 4 bits 3.4910 us/op 3.0060 us/op 1.16
RateTracker 1000000 limit, 1 obj count per request 220.52 ns/op 181.62 ns/op 1.21
RateTracker 1000000 limit, 2 obj count per request 166.04 ns/op 137.93 ns/op 1.20
RateTracker 1000000 limit, 4 obj count per request 139.18 ns/op 114.00 ns/op 1.22
RateTracker 1000000 limit, 8 obj count per request 124.54 ns/op 101.63 ns/op 1.23
RateTracker with prune 5.2290 us/op 3.7660 us/op 1.39
array of 16000 items push then shift 3.8125 us/op 3.1588 us/op 1.21
LinkedList of 16000 items push then shift 20.195 ns/op 17.363 ns/op 1.16
array of 16000 items push then pop 244.66 ns/op 207.32 ns/op 1.18
LinkedList of 16000 items push then pop 19.125 ns/op 17.108 ns/op 1.12
array of 24000 items push then shift 5.4576 us/op 4.5554 us/op 1.20
LinkedList of 24000 items push then shift 22.759 ns/op 21.348 ns/op 1.07
array of 24000 items push then pop 232.47 ns/op 185.25 ns/op 1.25
LinkedList of 24000 items push then pop 20.686 ns/op 19.248 ns/op 1.07

by benchmarkbot/action

disconnected++;
break;
default:
connecting++;
Copy link
Contributor

@dapplion dapplion Mar 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So if we have 1 peer with 20 open connections it will return connected: 20, is this the expected behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated to use the most relevant connection for a peer in the counting

@dadepo dadepo dismissed a stale review via 25a9c92 March 21, 2022 14:14
@dadepo
Copy link
Contributor Author

dadepo commented Mar 22, 2022

Trying the same endpoint on lighthouse after running for about 5 minutes returns count for both connected and disconnected, while for lodestar, the endpoint only returns count for connected and nothing for disconnected or the other status. Still need to see if the current way is the most accurate way of getting the connection info from libp2p.

Looked into trying to get disconnected peer by making a diff with the peers found in the peerstore and the one returned by the connection manager (reasoning is if it is found in the peerstore but not in the connection manager, then those are disconnected). The issue with that is the peerstore persists across restarts, meaning the result of the diff would keep on increasing.

Leaving the implementation for now to use the connection manager for all the counting. Open to other suggestions, if that is not fit enough

@dadepo dadepo marked this pull request as ready for review March 22, 2022 08:50
wemeetagain
wemeetagain previously approved these changes Mar 22, 2022
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I think its fine as is to just use the connection manager.

@@ -51,13 +51,34 @@ export function getNodeApi(opts: IApiOptions, {network, sync}: Pick<ApiModules,
},

async getPeerCount() {
// TODO: Implement
let disconnected = 0;
Copy link
Contributor

@dapplion dapplion Mar 23, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So disconnected means all peers that have ever connected to us and are no longer connected? That sounds like a memory leak to track lol. How does Lighthouse track this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup. Lighthouse has a network_globals object that tracks this. So when a peer is disconnected, it is tracked, as can be seen here here which is then read and used to return the peer count as can be seen here only that the network_globals is not persistent.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's persisted to disk that's how it doesn't leak memory. I would leave a // TODO: Implement disconnect count with on-disk persistence

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed it is not being persisted as the values don't persist after each restarts. Will dig later to see how that is implemented...

I would leave a // TODO: Implement disconnect count with on-disk persistence

Ok. Will do.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will dig later to see how that is implemented...

I would ignore completely, tracking disconnected peers is an extremely unnecessary feature in my opinion haha

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Beacon API peer_count call always returns 0
4 participants