Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chore: add geth metrics #5510

Merged
merged 1 commit into from
May 28, 2023
Merged

chore: add geth metrics #5510

merged 1 commit into from
May 28, 2023

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented May 18, 2023

Motivation

We want to compare lodestar's execution engine call duration with geth's metric

Description

Add geth metrics to Execution Engine dashboard

Screenshot 2023-05-18 at 11 31 06

cc @dapplion

@twoeths twoeths requested a review from a team as a code owner May 18, 2023 04:31
@github-actions
Copy link
Contributor

github-actions bot commented May 18, 2023 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 6be1972 Previous: 118f7b9 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.0402 ms/op 645.00 us/op 1.61
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 45.266 us/op 50.532 us/op 0.90
BLS verify - blst-native 1.2073 ms/op 1.2417 ms/op 0.97
BLS verifyMultipleSignatures 3 - blst-native 2.3860 ms/op 2.5166 ms/op 0.95
BLS verifyMultipleSignatures 8 - blst-native 5.1169 ms/op 5.4125 ms/op 0.95
BLS verifyMultipleSignatures 32 - blst-native 18.586 ms/op 19.570 ms/op 0.95
BLS aggregatePubkeys 32 - blst-native 24.920 us/op 26.032 us/op 0.96
BLS aggregatePubkeys 128 - blst-native 96.860 us/op 102.21 us/op 0.95
getAttestationsForBlock 62.484 ms/op 56.477 ms/op 1.11
isKnown best case - 1 super set check 258.00 ns/op 266.00 ns/op 0.97
isKnown normal case - 2 super set checks 248.00 ns/op 252.00 ns/op 0.98
isKnown worse case - 16 super set checks 250.00 ns/op 246.00 ns/op 1.02
CheckpointStateCache - add get delete 4.8630 us/op 5.4830 us/op 0.89
validate gossip signedAggregateAndProof - struct 2.6765 ms/op 2.8301 ms/op 0.95
validate gossip attestation - struct 1.2756 ms/op 1.3854 ms/op 0.92
pickEth1Vote - no votes 1.2155 ms/op 1.2989 ms/op 0.94
pickEth1Vote - max votes 11.606 ms/op 10.221 ms/op 1.14
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.5532 ms/op 9.2327 ms/op 0.93
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 17.861 ms/op 14.044 ms/op 1.27
pickEth1Vote - Eth1Data fastSerialize value x2048 629.24 us/op 659.13 us/op 0.95
pickEth1Vote - Eth1Data fastSerialize tree x2048 7.9878 ms/op 4.7311 ms/op 1.69
bytes32 toHexString 481.00 ns/op 523.00 ns/op 0.92
bytes32 Buffer.toString(hex) 352.00 ns/op 347.00 ns/op 1.01
bytes32 Buffer.toString(hex) from Uint8Array 548.00 ns/op 595.00 ns/op 0.92
bytes32 Buffer.toString(hex) + 0x 350.00 ns/op 372.00 ns/op 0.94
Object access 1 prop 0.15900 ns/op 0.19000 ns/op 0.84
Map access 1 prop 0.16800 ns/op 0.16600 ns/op 1.01
Object get x1000 6.5880 ns/op 6.6520 ns/op 0.99
Map get x1000 0.58400 ns/op 0.60800 ns/op 0.96
Object set x1000 49.777 ns/op 53.862 ns/op 0.92
Map set x1000 43.378 ns/op 44.386 ns/op 0.98
Return object 10000 times 0.23300 ns/op 0.24400 ns/op 0.95
Throw Error 10000 times 4.1599 us/op 4.3004 us/op 0.97
fastMsgIdFn sha256 / 200 bytes 3.3880 us/op 3.5640 us/op 0.95
fastMsgIdFn h32 xxhash / 200 bytes 279.00 ns/op 300.00 ns/op 0.93
fastMsgIdFn h64 xxhash / 200 bytes 390.00 ns/op 420.00 ns/op 0.93
fastMsgIdFn sha256 / 1000 bytes 11.450 us/op 12.018 us/op 0.95
fastMsgIdFn h32 xxhash / 1000 bytes 402.00 ns/op 414.00 ns/op 0.97
fastMsgIdFn h64 xxhash / 1000 bytes 462.00 ns/op 492.00 ns/op 0.94
fastMsgIdFn sha256 / 10000 bytes 102.79 us/op 104.83 us/op 0.98
fastMsgIdFn h32 xxhash / 10000 bytes 1.8790 us/op 1.9510 us/op 0.96
fastMsgIdFn h64 xxhash / 10000 bytes 1.3260 us/op 1.4330 us/op 0.93
enrSubnets - fastDeserialize 64 bits 1.2500 us/op 1.3990 us/op 0.89
enrSubnets - ssz BitVector 64 bits 481.00 ns/op 526.00 ns/op 0.91
enrSubnets - fastDeserialize 4 bits 171.00 ns/op 190.00 ns/op 0.90
enrSubnets - ssz BitVector 4 bits 487.00 ns/op 539.00 ns/op 0.90
prioritizePeers score -10:0 att 32-0.1 sync 2-0 103.50 us/op 117.97 us/op 0.88
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 128.11 us/op 142.14 us/op 0.90
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 161.34 us/op 187.01 us/op 0.86
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 291.00 us/op 340.68 us/op 0.85
prioritizePeers score 0:0 att 64-1 sync 4-1 356.36 us/op 394.25 us/op 0.90
array of 16000 items push then shift 1.5927 us/op 1.6654 us/op 0.96
LinkedList of 16000 items push then shift 8.7390 ns/op 9.0300 ns/op 0.97
array of 16000 items push then pop 72.933 ns/op 111.08 ns/op 0.66
LinkedList of 16000 items push then pop 8.5040 ns/op 8.8840 ns/op 0.96
array of 24000 items push then shift 2.3332 us/op 2.4288 us/op 0.96
LinkedList of 24000 items push then shift 10.662 ns/op 9.2030 ns/op 1.16
array of 24000 items push then pop 76.976 ns/op 88.281 ns/op 0.87
LinkedList of 24000 items push then pop 8.2120 ns/op 8.9620 ns/op 0.92
intersect bitArray bitLen 8 12.952 ns/op 13.851 ns/op 0.94
intersect array and set length 8 72.641 ns/op 101.97 ns/op 0.71
intersect bitArray bitLen 128 42.609 ns/op 45.646 ns/op 0.93
intersect array and set length 128 1.0019 us/op 1.1651 us/op 0.86
Buffer.concat 32 items 2.5950 us/op 3.1870 us/op 0.81
Uint8Array.set 32 items 2.8230 us/op 3.2070 us/op 0.88
pass gossip attestations to forkchoice per slot 2.9145 ms/op 2.9799 ms/op 0.98
computeDeltas 2.8838 ms/op 3.1512 ms/op 0.92
computeProposerBoostScoreFromBalances 1.7330 ms/op 1.8154 ms/op 0.95
altair processAttestation - 250000 vs - 7PWei normalcase 2.1399 ms/op 2.2126 ms/op 0.97
altair processAttestation - 250000 vs - 7PWei worstcase 3.2360 ms/op 4.3670 ms/op 0.74
altair processAttestation - setStatus - 1/6 committees join 132.13 us/op 145.24 us/op 0.91
altair processAttestation - setStatus - 1/3 committees join 264.40 us/op 293.89 us/op 0.90
altair processAttestation - setStatus - 1/2 committees join 357.83 us/op 437.64 us/op 0.82
altair processAttestation - setStatus - 2/3 committees join 442.23 us/op 478.63 us/op 0.92
altair processAttestation - setStatus - 4/5 committees join 614.32 us/op 674.80 us/op 0.91
altair processAttestation - setStatus - 100% committees join 750.43 us/op 811.27 us/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase 22.442 ms/op 17.025 ms/op 1.32
altair processBlock - 250000 vs - 7PWei normalcase hashState 32.033 ms/op 25.174 ms/op 1.27
altair processBlock - 250000 vs - 7PWei worstcase 62.139 ms/op 54.334 ms/op 1.14
altair processBlock - 250000 vs - 7PWei worstcase hashState 76.166 ms/op 71.365 ms/op 1.07
phase0 processBlock - 250000 vs - 7PWei normalcase 1.8929 ms/op 2.3557 ms/op 0.80
phase0 processBlock - 250000 vs - 7PWei worstcase 27.288 ms/op 32.155 ms/op 0.85
altair processEth1Data - 250000 vs - 7PWei normalcase 453.09 us/op 637.63 us/op 0.71
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 10.609 us/op 11.562 us/op 0.92
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 19.231 us/op 25.363 us/op 0.76
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 13.413 us/op 10.313 us/op 1.30
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 10.296 us/op 8.1230 us/op 1.27
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 73.797 us/op 104.22 us/op 0.71
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.2243 ms/op 714.23 us/op 1.71
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.6503 ms/op 914.93 us/op 1.80
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.6125 ms/op 913.89 us/op 1.76
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 2.1504 ms/op 2.3729 ms/op 0.91
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.2810 ms/op 1.7319 ms/op 1.32
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.8291 ms/op 4.0726 ms/op 0.94
Tree 40 250000 create 319.95 ms/op 334.38 ms/op 0.96
Tree 40 250000 get(125000) 177.04 ns/op 192.04 ns/op 0.92
Tree 40 250000 set(125000) 939.77 ns/op 1.0172 us/op 0.92
Tree 40 250000 toArray() 17.964 ms/op 19.596 ms/op 0.92
Tree 40 250000 iterate all - toArray() + loop 16.528 ms/op 21.307 ms/op 0.78
Tree 40 250000 iterate all - get(i) 66.604 ms/op 73.294 ms/op 0.91
MutableVector 250000 create 13.170 ms/op 10.540 ms/op 1.25
MutableVector 250000 get(125000) 6.4020 ns/op 7.2430 ns/op 0.88
MutableVector 250000 set(125000) 261.57 ns/op 256.46 ns/op 1.02
MutableVector 250000 toArray() 3.0207 ms/op 2.7603 ms/op 1.09
MutableVector 250000 iterate all - toArray() + loop 2.9124 ms/op 3.0665 ms/op 0.95
MutableVector 250000 iterate all - get(i) 2.2016 ms/op 1.7405 ms/op 1.26
Array 250000 create 2.5809 ms/op 2.7465 ms/op 0.94
Array 250000 clone - spread 1.1074 ms/op 1.1835 ms/op 0.94
Array 250000 get(125000) 0.54200 ns/op 0.58400 ns/op 0.93
Array 250000 set(125000) 0.61800 ns/op 0.66300 ns/op 0.93
Array 250000 iterate all - loop 79.817 us/op 88.404 us/op 0.90
effectiveBalanceIncrements clone Uint8Array 300000 24.423 us/op 35.693 us/op 0.68
effectiveBalanceIncrements clone MutableVector 300000 351.00 ns/op 400.00 ns/op 0.88
effectiveBalanceIncrements rw all Uint8Array 300000 164.79 us/op 169.43 us/op 0.97
effectiveBalanceIncrements rw all MutableVector 300000 80.344 ms/op 92.619 ms/op 0.87
phase0 afterProcessEpoch - 250000 vs - 7PWei 111.94 ms/op 117.10 ms/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 42.514 ms/op 41.452 ms/op 1.03
altair processEpoch - mainnet_e81889 350.77 ms/op 358.35 ms/op 0.98
mainnet_e81889 - altair beforeProcessEpoch 67.206 ms/op 89.756 ms/op 0.75
mainnet_e81889 - altair processJustificationAndFinalization 24.617 us/op 27.113 us/op 0.91
mainnet_e81889 - altair processInactivityUpdates 5.5697 ms/op 7.0442 ms/op 0.79
mainnet_e81889 - altair processRewardsAndPenalties 78.837 ms/op 64.241 ms/op 1.23
mainnet_e81889 - altair processRegistryUpdates 4.6650 us/op 4.7590 us/op 0.98
mainnet_e81889 - altair processSlashings 814.00 ns/op 988.00 ns/op 0.82
mainnet_e81889 - altair processEth1DataReset 828.00 ns/op 923.00 ns/op 0.90
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.2853 ms/op 1.4421 ms/op 1.58
mainnet_e81889 - altair processSlashingsReset 7.4880 us/op 8.1150 us/op 0.92
mainnet_e81889 - altair processRandaoMixesReset 7.6240 us/op 9.3650 us/op 0.81
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0110 us/op 1.8020 us/op 0.56
mainnet_e81889 - altair processParticipationFlagUpdates 2.8730 us/op 5.0800 us/op 0.57
mainnet_e81889 - altair processSyncCommitteeUpdates 748.00 ns/op 1.5570 us/op 0.48
mainnet_e81889 - altair afterProcessEpoch 124.77 ms/op 131.72 ms/op 0.95
phase0 processEpoch - mainnet_e58758 398.77 ms/op 432.50 ms/op 0.92
mainnet_e58758 - phase0 beforeProcessEpoch 141.36 ms/op 139.47 ms/op 1.01
mainnet_e58758 - phase0 processJustificationAndFinalization 26.456 us/op 27.554 us/op 0.96
mainnet_e58758 - phase0 processRewardsAndPenalties 66.176 ms/op 79.418 ms/op 0.83
mainnet_e58758 - phase0 processRegistryUpdates 10.982 us/op 15.474 us/op 0.71
mainnet_e58758 - phase0 processSlashings 788.00 ns/op 1.3260 us/op 0.59
mainnet_e58758 - phase0 processEth1DataReset 883.00 ns/op 1.3740 us/op 0.64
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.8412 ms/op 1.9462 ms/op 0.95
mainnet_e58758 - phase0 processSlashingsReset 5.5030 us/op 7.9220 us/op 0.69
mainnet_e58758 - phase0 processRandaoMixesReset 7.6680 us/op 11.320 us/op 0.68
mainnet_e58758 - phase0 processHistoricalRootsUpdate 1.3950 us/op 1.8220 us/op 0.77
mainnet_e58758 - phase0 processParticipationRecordUpdates 5.4490 us/op 10.043 us/op 0.54
mainnet_e58758 - phase0 afterProcessEpoch 104.72 ms/op 124.61 ms/op 0.84
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.2308 ms/op 2.1224 ms/op 0.58
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.8348 ms/op 3.4400 ms/op 0.53
altair processInactivityUpdates - 250000 normalcase 24.885 ms/op 31.842 ms/op 0.78
altair processInactivityUpdates - 250000 worstcase 27.872 ms/op 30.279 ms/op 0.92
phase0 processRegistryUpdates - 250000 normalcase 6.9890 us/op 8.4670 us/op 0.83
phase0 processRegistryUpdates - 250000 badcase_full_deposits 314.47 us/op 294.33 us/op 1.07
phase0 processRegistryUpdates - 250000 worstcase 0.5 133.59 ms/op 112.04 ms/op 1.19
altair processRewardsAndPenalties - 250000 normalcase 68.775 ms/op 64.931 ms/op 1.06
altair processRewardsAndPenalties - 250000 worstcase 77.707 ms/op 69.922 ms/op 1.11
phase0 getAttestationDeltas - 250000 normalcase 6.8118 ms/op 9.3365 ms/op 0.73
phase0 getAttestationDeltas - 250000 worstcase 6.5931 ms/op 7.2126 ms/op 0.91
phase0 processSlashings - 250000 worstcase 3.2252 ms/op 3.9329 ms/op 0.82
altair processSyncCommitteeUpdates - 250000 183.03 ms/op 196.25 ms/op 0.93
BeaconState.hashTreeRoot - No change 270.00 ns/op 301.00 ns/op 0.90
BeaconState.hashTreeRoot - 1 full validator 55.687 us/op 54.889 us/op 1.01
BeaconState.hashTreeRoot - 32 full validator 695.40 us/op 567.99 us/op 1.22
BeaconState.hashTreeRoot - 512 full validator 4.8562 ms/op 5.5610 ms/op 0.87
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 61.541 us/op 66.016 us/op 0.93
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.0276 ms/op 1.0342 ms/op 0.99
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.271 ms/op 12.983 ms/op 0.87
BeaconState.hashTreeRoot - 1 balances 62.789 us/op 52.538 us/op 1.20
BeaconState.hashTreeRoot - 32 balances 554.60 us/op 504.39 us/op 1.10
BeaconState.hashTreeRoot - 512 balances 4.6466 ms/op 4.5113 ms/op 1.03
BeaconState.hashTreeRoot - 250000 balances 80.118 ms/op 76.428 ms/op 1.05
aggregationBits - 2048 els - zipIndexesInBitList 15.526 us/op 19.541 us/op 0.79
regular array get 100000 times 34.402 us/op 34.227 us/op 1.01
wrappedArray get 100000 times 32.053 us/op 34.320 us/op 0.93
arrayWithProxy get 100000 times 16.858 ms/op 18.737 ms/op 0.90
ssz.Root.equals 537.00 ns/op 606.00 ns/op 0.89
byteArrayEquals 534.00 ns/op 607.00 ns/op 0.88
shuffle list - 16384 els 6.7974 ms/op 7.5681 ms/op 0.90
shuffle list - 250000 els 99.716 ms/op 109.07 ms/op 0.91
processSlot - 1 slots 8.5900 us/op 10.575 us/op 0.81
processSlot - 32 slots 1.4461 ms/op 1.5087 ms/op 0.96
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 41.658 ms/op 40.281 ms/op 1.03
getCommitteeAssignments - req 1 vs - 250000 vc 2.9064 ms/op 3.3873 ms/op 0.86
getCommitteeAssignments - req 100 vs - 250000 vc 4.1192 ms/op 4.7433 ms/op 0.87
getCommitteeAssignments - req 1000 vs - 250000 vc 4.4016 ms/op 5.1628 ms/op 0.85
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.6600 ns/op 5.8700 ns/op 0.79
state getBlockRootAtSlot - 250000 vs - 7PWei 552.81 ns/op 1.1918 us/op 0.46
computeProposers - vc 250000 10.455 ms/op 12.115 ms/op 0.86
computeEpochShuffling - vc 250000 101.33 ms/op 109.50 ms/op 0.93
getNextSyncCommittee - vc 250000 176.89 ms/op 198.08 ms/op 0.89
computeSigningRoot for AttestationData 13.280 us/op 14.754 us/op 0.90
hash AttestationData serialized data then Buffer.toString(base64) 2.3973 us/op 2.6448 us/op 0.91
toHexString serialized data 1.0980 us/op 1.1899 us/op 0.92
Buffer.toString(base64) 325.81 ns/op 375.61 ns/op 0.87

by benchmarkbot/action

@dapplion
Copy link
Contributor

@tuyennhv why the huge diff?

@twoeths
Copy link
Contributor Author

twoeths commented May 19, 2023

@dapplion I don't know how github shows this huge diff, here's in my VsCode

Screenshot 2023-05-19 at 14 50 12

main change is from line 1476 to 1934

@twoeths
Copy link
Contributor Author

twoeths commented May 25, 2023

updated the Dashboard following #5525 thanks @dapplion 🚀

@twoeths twoeths changed the title feat: add geth metrics chore: add geth metrics May 25, 2023
@twoeths twoeths merged commit 8fa4b1f into unstable May 28, 2023
@twoeths twoeths deleted the tuyen/geth_metrics branch May 28, 2023 05:36
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.9.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewkeil matthewkeil matthewkeil left review comments

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@twoeths @dapplion @wemeetagain @matthewkeil