Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add vc bn fallback #4648

Merged
merged 12 commits into from
Oct 10, 2022
Merged

Add vc bn fallback #4648

merged 12 commits into from
Oct 10, 2022

Conversation

dapplion
Copy link
Contributor

@dapplion dapplion commented Oct 6, 2022

Motivation

Description

A good vc bn fallback must fulfill these goals

  • if first server is stable and responding do not query fallbacks
  • if first server errors, retry that same request on fallbacks
  • until first server is shown to be reliable again, contact all servers

This behaviour is not simple, so classic patterns like Promise.all() or for try catch are not enough.

The solution of this PR appears complex, but that's what's needed to achieve the goals above. Please check the inline comments for pointers on the logic.

A unit tests proves that it works, but I haven't tested yet on a deployed beacon node.

Closes #4337

@dapplion dapplion requested a review from a team as a code owner October 6, 2022 23:43
wemeetagain
wemeetagain previously approved these changes Oct 7, 2022
Copy link
Member

@wemeetagain wemeetagain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be able to use Promise.any, maybe not worth it tho :P
looks good

@dapplion
Copy link
Contributor Author

dapplion commented Oct 7, 2022

You might be able to use Promise.any, maybe not worth it tho :P

Looked into it and doesn't simplify the code that much since I have to track which URL completed the request successfully. Also Promise.any is available in NodeJS >= 15.0.0 and our engine is still > 12.9

@github-actions
Copy link
Contributor

github-actions bot commented Oct 7, 2022

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: a929d78 Previous: 1cba836 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.4231 ms/op 2.0390 ms/op 1.19
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 73.356 us/op 71.259 us/op 1.03
BLS verify - blst-native 1.8564 ms/op 1.8581 ms/op 1.00
BLS verifyMultipleSignatures 3 - blst-native 3.8030 ms/op 3.8009 ms/op 1.00
BLS verifyMultipleSignatures 8 - blst-native 8.1853 ms/op 8.1881 ms/op 1.00
BLS verifyMultipleSignatures 32 - blst-native 29.677 ms/op 29.660 ms/op 1.00
BLS aggregatePubkeys 32 - blst-native 39.245 us/op 39.480 us/op 0.99
BLS aggregatePubkeys 128 - blst-native 153.29 us/op 152.98 us/op 1.00
getAttestationsForBlock 90.110 ms/op 86.630 ms/op 1.04
isKnown best case - 1 super set check 426.00 ns/op 465.00 ns/op 0.92
isKnown normal case - 2 super set checks 412.00 ns/op 449.00 ns/op 0.92
isKnown worse case - 16 super set checks 418.00 ns/op 459.00 ns/op 0.91
CheckpointStateCache - add get delete 9.1430 us/op 8.5490 us/op 1.07
validate gossip signedAggregateAndProof - struct 4.2750 ms/op 4.2813 ms/op 1.00
validate gossip attestation - struct 2.0365 ms/op 2.0414 ms/op 1.00
pickEth1Vote - no votes 2.1861 ms/op 2.1860 ms/op 1.00
pickEth1Vote - max votes 21.294 ms/op 19.449 ms/op 1.09
pickEth1Vote - Eth1Data hashTreeRoot value x2048 11.452 ms/op 11.024 ms/op 1.04
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.925 ms/op 20.022 ms/op 1.05
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5552 ms/op 1.5501 ms/op 1.00
pickEth1Vote - Eth1Data fastSerialize tree x2048 15.115 ms/op 13.109 ms/op 1.15
bytes32 toHexString 1.1160 us/op 1.0540 us/op 1.06
bytes32 Buffer.toString(hex) 754.00 ns/op 692.00 ns/op 1.09
bytes32 Buffer.toString(hex) from Uint8Array 943.00 ns/op 989.00 ns/op 0.95
bytes32 Buffer.toString(hex) + 0x 730.00 ns/op 684.00 ns/op 1.07
Object access 1 prop 0.37500 ns/op 0.34900 ns/op 1.07
Map access 1 prop 0.29000 ns/op 0.30100 ns/op 0.96
Object get x1000 18.058 ns/op 17.896 ns/op 1.01
Map get x1000 0.97300 ns/op 0.98500 ns/op 0.99
Object set x1000 124.98 ns/op 117.29 ns/op 1.07
Map set x1000 72.990 ns/op 69.644 ns/op 1.05
Return object 10000 times 0.37520 ns/op 0.36590 ns/op 1.03
Throw Error 10000 times 5.9801 us/op 5.8656 us/op 1.02
fastMsgIdFn sha256 / 200 bytes 4.2210 us/op 4.0830 us/op 1.03
fastMsgIdFn h32 xxhash / 200 bytes 539.00 ns/op 514.00 ns/op 1.05
fastMsgIdFn h64 xxhash / 200 bytes 670.00 ns/op 736.00 ns/op 0.91
fastMsgIdFn sha256 / 1000 bytes 12.957 us/op 13.074 us/op 0.99
fastMsgIdFn h32 xxhash / 1000 bytes 691.00 ns/op 681.00 ns/op 1.01
fastMsgIdFn h64 xxhash / 1000 bytes 820.00 ns/op 805.00 ns/op 1.02
fastMsgIdFn sha256 / 10000 bytes 113.19 us/op 112.23 us/op 1.01
fastMsgIdFn h32 xxhash / 10000 bytes 2.4160 us/op 2.2630 us/op 1.07
fastMsgIdFn h64 xxhash / 10000 bytes 1.7680 us/op 1.7530 us/op 1.01
enrSubnets - fastDeserialize 64 bits 2.7430 us/op 2.5660 us/op 1.07
enrSubnets - ssz BitVector 64 bits 765.00 ns/op 729.00 ns/op 1.05
enrSubnets - fastDeserialize 4 bits 399.00 ns/op 378.00 ns/op 1.06
enrSubnets - ssz BitVector 4 bits 783.00 ns/op 767.00 ns/op 1.02
prioritizePeers score -10:0 att 32-0.1 sync 2-0 97.304 us/op 92.016 us/op 1.06
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 136.10 us/op 123.32 us/op 1.10
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 237.29 us/op 212.86 us/op 1.11
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 393.81 us/op 452.28 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 457.46 us/op 457.84 us/op 1.00
RateTracker 1000000 limit, 1 obj count per request 189.47 ns/op 205.96 ns/op 0.92
RateTracker 1000000 limit, 2 obj count per request 140.07 ns/op 152.76 ns/op 0.92
RateTracker 1000000 limit, 4 obj count per request 115.81 ns/op 128.88 ns/op 0.90
RateTracker 1000000 limit, 8 obj count per request 106.31 ns/op 117.37 ns/op 0.91
RateTracker with prune 4.3740 us/op 4.2030 us/op 1.04
array of 16000 items push then shift 3.0841 us/op 3.1905 us/op 0.97
LinkedList of 16000 items push then shift 17.080 ns/op 16.934 ns/op 1.01
array of 16000 items push then pop 233.43 ns/op 240.89 ns/op 0.97
LinkedList of 16000 items push then pop 16.571 ns/op 16.797 ns/op 0.99
array of 24000 items push then shift 4.5506 us/op 4.5760 us/op 0.99
LinkedList of 24000 items push then shift 20.691 ns/op 19.986 ns/op 1.04
array of 24000 items push then pop 207.46 ns/op 209.31 ns/op 0.99
LinkedList of 24000 items push then pop 17.972 ns/op 17.901 ns/op 1.00
intersect bitArray bitLen 8 11.734 ns/op 13.415 ns/op 0.87
intersect array and set length 8 172.06 ns/op 156.84 ns/op 1.10
intersect bitArray bitLen 128 72.110 ns/op 61.940 ns/op 1.16
intersect array and set length 128 2.3268 us/op 2.3414 us/op 0.99
Buffer.concat 32 items 1.8650 ns/op 2.1530 ns/op 0.87
pass gossip attestations to forkchoice per slot 6.3926 ms/op 4.1222 ms/op 1.55
computeDeltas 5.9397 ms/op 5.5228 ms/op 1.08
computeProposerBoostScoreFromBalances 908.48 us/op 920.99 us/op 0.99
altair processAttestation - 250000 vs - 7PWei normalcase 4.1482 ms/op 3.6041 ms/op 1.15
altair processAttestation - 250000 vs - 7PWei worstcase 6.0775 ms/op 6.1835 ms/op 0.98
altair processAttestation - setStatus - 1/6 committees join 206.64 us/op 205.72 us/op 1.00
altair processAttestation - setStatus - 1/3 committees join 397.70 us/op 396.80 us/op 1.00
altair processAttestation - setStatus - 1/2 committees join 554.39 us/op 553.95 us/op 1.00
altair processAttestation - setStatus - 2/3 committees join 720.91 us/op 713.34 us/op 1.01
altair processAttestation - setStatus - 4/5 committees join 991.42 us/op 988.35 us/op 1.00
altair processAttestation - setStatus - 100% committees join 1.1792 ms/op 1.1789 ms/op 1.00
altair processBlock - 250000 vs - 7PWei normalcase 26.853 ms/op 27.042 ms/op 0.99
altair processBlock - 250000 vs - 7PWei normalcase hashState 36.858 ms/op 38.085 ms/op 0.97
altair processBlock - 250000 vs - 7PWei worstcase 81.691 ms/op 77.869 ms/op 1.05
altair processBlock - 250000 vs - 7PWei worstcase hashState 97.419 ms/op 95.194 ms/op 1.02
phase0 processBlock - 250000 vs - 7PWei normalcase 4.4794 ms/op 3.6274 ms/op 1.23
phase0 processBlock - 250000 vs - 7PWei worstcase 48.977 ms/op 46.153 ms/op 1.06
altair processEth1Data - 250000 vs - 7PWei normalcase 817.23 us/op 831.46 us/op 0.98
Tree 40 250000 create 877.51 ms/op 737.55 ms/op 1.19
Tree 40 250000 get(125000) 293.95 ns/op 282.64 ns/op 1.04
Tree 40 250000 set(125000) 2.7634 us/op 2.2515 us/op 1.23
Tree 40 250000 toArray() 32.641 ms/op 31.368 ms/op 1.04
Tree 40 250000 iterate all - toArray() + loop 32.731 ms/op 31.712 ms/op 1.03
Tree 40 250000 iterate all - get(i) 113.24 ms/op 108.25 ms/op 1.05
MutableVector 250000 create 16.747 ms/op 14.680 ms/op 1.14
MutableVector 250000 get(125000) 13.184 ns/op 14.771 ns/op 0.89
MutableVector 250000 set(125000) 715.66 ns/op 575.90 ns/op 1.24
MutableVector 250000 toArray() 7.8155 ms/op 7.1580 ms/op 1.09
MutableVector 250000 iterate all - toArray() + loop 7.0357 ms/op 7.4115 ms/op 0.95
MutableVector 250000 iterate all - get(i) 3.2864 ms/op 3.8262 ms/op 0.86
Array 250000 create 6.3366 ms/op 6.2032 ms/op 1.02
Array 250000 clone - spread 2.6472 ms/op 2.5071 ms/op 1.06
Array 250000 get(125000) 1.1420 ns/op 1.0870 ns/op 1.05
Array 250000 set(125000) 1.1320 ns/op 1.0830 ns/op 1.05
Array 250000 iterate all - loop 167.81 us/op 168.98 us/op 0.99
effectiveBalanceIncrements clone Uint8Array 300000 76.487 us/op 74.961 us/op 1.02
effectiveBalanceIncrements clone MutableVector 300000 788.00 ns/op 694.00 ns/op 1.14
effectiveBalanceIncrements rw all Uint8Array 300000 252.76 us/op 253.95 us/op 1.00
effectiveBalanceIncrements rw all MutableVector 300000 189.13 ms/op 159.91 ms/op 1.18
phase0 afterProcessEpoch - 250000 vs - 7PWei 184.09 ms/op 184.67 ms/op 1.00
phase0 beforeProcessEpoch - 250000 vs - 7PWei 72.291 ms/op 68.660 ms/op 1.05
altair processEpoch - mainnet_e81889 499.62 ms/op 576.94 ms/op 0.87
mainnet_e81889 - altair beforeProcessEpoch 150.23 ms/op 155.45 ms/op 0.97
mainnet_e81889 - altair processJustificationAndFinalization 20.678 us/op 23.852 us/op 0.87
mainnet_e81889 - altair processInactivityUpdates 10.498 ms/op 10.705 ms/op 0.98
mainnet_e81889 - altair processRewardsAndPenalties 91.157 ms/op 89.698 ms/op 1.02
mainnet_e81889 - altair processRegistryUpdates 3.2510 us/op 3.5880 us/op 0.91
mainnet_e81889 - altair processSlashings 624.00 ns/op 791.00 ns/op 0.79
mainnet_e81889 - altair processEth1DataReset 787.00 ns/op 944.00 ns/op 0.83
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.3039 ms/op 2.4026 ms/op 0.96
mainnet_e81889 - altair processSlashingsReset 5.9130 us/op 6.4450 us/op 0.92
mainnet_e81889 - altair processRandaoMixesReset 5.3750 us/op 6.8710 us/op 0.78
mainnet_e81889 - altair processHistoricalRootsUpdate 837.00 ns/op 911.00 ns/op 0.92
mainnet_e81889 - altair processParticipationFlagUpdates 2.9000 us/op 3.3080 us/op 0.88
mainnet_e81889 - altair processSyncCommitteeUpdates 632.00 ns/op 815.00 ns/op 0.78
mainnet_e81889 - altair afterProcessEpoch 195.56 ms/op 196.08 ms/op 1.00
phase0 processEpoch - mainnet_e58758 526.84 ms/op 521.53 ms/op 1.01
mainnet_e58758 - phase0 beforeProcessEpoch 229.94 ms/op 224.74 ms/op 1.02
mainnet_e58758 - phase0 processJustificationAndFinalization 20.064 us/op 24.448 us/op 0.82
mainnet_e58758 - phase0 processRewardsAndPenalties 139.19 ms/op 123.97 ms/op 1.12
mainnet_e58758 - phase0 processRegistryUpdates 9.1740 us/op 10.565 us/op 0.87
mainnet_e58758 - phase0 processSlashings 744.00 ns/op 752.00 ns/op 0.99
mainnet_e58758 - phase0 processEth1DataReset 794.00 ns/op 923.00 ns/op 0.86
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.9003 ms/op 2.2146 ms/op 0.86
mainnet_e58758 - phase0 processSlashingsReset 4.7870 us/op 5.8220 us/op 0.82
mainnet_e58758 - phase0 processRandaoMixesReset 5.6030 us/op 6.5040 us/op 0.86
mainnet_e58758 - phase0 processHistoricalRootsUpdate 852.00 ns/op 846.00 ns/op 1.01
mainnet_e58758 - phase0 processParticipationRecordUpdates 4.7630 us/op 5.4020 us/op 0.88
mainnet_e58758 - phase0 afterProcessEpoch 160.46 ms/op 160.27 ms/op 1.00
phase0 processEffectiveBalanceUpdates - 250000 normalcase 2.6350 ms/op 2.5525 ms/op 1.03
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 3.0864 ms/op 3.4728 ms/op 0.89
altair processInactivityUpdates - 250000 normalcase 41.067 ms/op 38.539 ms/op 1.07
altair processInactivityUpdates - 250000 worstcase 46.655 ms/op 39.953 ms/op 1.17
phase0 processRegistryUpdates - 250000 normalcase 7.7500 us/op 8.6900 us/op 0.89
phase0 processRegistryUpdates - 250000 badcase_full_deposits 401.72 us/op 407.76 us/op 0.99
phase0 processRegistryUpdates - 250000 worstcase 0.5 205.17 ms/op 200.11 ms/op 1.03
altair processRewardsAndPenalties - 250000 normalcase 128.77 ms/op 85.718 ms/op 1.50
altair processRewardsAndPenalties - 250000 worstcase 84.101 ms/op 133.58 ms/op 0.63
phase0 getAttestationDeltas - 250000 normalcase 12.947 ms/op 12.274 ms/op 1.05
phase0 getAttestationDeltas - 250000 worstcase 13.615 ms/op 12.353 ms/op 1.10
phase0 processSlashings - 250000 worstcase 5.3814 ms/op 5.3915 ms/op 1.00
altair processSyncCommitteeUpdates - 250000 281.42 ms/op 277.87 ms/op 1.01
BeaconState.hashTreeRoot - No change 495.00 ns/op 504.00 ns/op 0.98
BeaconState.hashTreeRoot - 1 full validator 54.968 us/op 58.795 us/op 0.93
BeaconState.hashTreeRoot - 32 full validator 676.77 us/op 543.18 us/op 1.25
BeaconState.hashTreeRoot - 512 full validator 5.9131 ms/op 6.5673 ms/op 0.90
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 79.304 us/op 78.259 us/op 1.01
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.1543 ms/op 1.1509 ms/op 1.00
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 17.948 ms/op 15.379 ms/op 1.17
BeaconState.hashTreeRoot - 1 balances 59.770 us/op 61.371 us/op 0.97
BeaconState.hashTreeRoot - 32 balances 549.91 us/op 593.96 us/op 0.93
BeaconState.hashTreeRoot - 512 balances 5.7385 ms/op 6.0341 ms/op 0.95
BeaconState.hashTreeRoot - 250000 balances 85.960 ms/op 86.820 ms/op 0.99
aggregationBits - 2048 els - zipIndexesInBitList 38.130 us/op 30.626 us/op 1.25
regular array get 100000 times 67.451 us/op 67.464 us/op 1.00
wrappedArray get 100000 times 67.465 us/op 67.466 us/op 1.00
arrayWithProxy get 100000 times 29.296 ms/op 29.047 ms/op 1.01
ssz.Root.equals 2.4490 us/op 490.00 ns/op 5.00
byteArrayEquals 501.00 ns/op 501.00 ns/op 1.00
shuffle list - 16384 els 11.181 ms/op 11.209 ms/op 1.00
shuffle list - 250000 els 164.48 ms/op 165.20 ms/op 1.00
processSlot - 1 slots 12.128 us/op 12.652 us/op 0.96
processSlot - 32 slots 1.7217 ms/op 1.7360 ms/op 0.99
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 373.14 us/op 390.57 us/op 0.96
getCommitteeAssignments - req 1 vs - 250000 vc 5.2896 ms/op 5.3288 ms/op 0.99
getCommitteeAssignments - req 100 vs - 250000 vc 7.3245 ms/op 7.2962 ms/op 1.00
getCommitteeAssignments - req 1000 vs - 250000 vc 7.7692 ms/op 7.7862 ms/op 1.00
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 9.6900 ns/op 10.220 ns/op 0.95
state getBlockRootAtSlot - 250000 vs - 7PWei 1.2850 us/op 1.1259 us/op 1.14
computeProposers - vc 250000 16.508 ms/op 16.354 ms/op 1.01
computeEpochShuffling - vc 250000 168.15 ms/op 169.15 ms/op 0.99
getNextSyncCommittee - vc 250000 268.85 ms/op 269.89 ms/op 1.00

by benchmarkbot/action

@philknows
Copy link
Member

Is there any sort of notification for us we can use when a fallback is triggered?

@dapplion
Copy link
Contributor Author

dapplion commented Oct 7, 2022

Is there any sort of notification for us we can use when a fallback is triggered?

Yes, this PRs adds metrics that will clearly show if a URL is down

resolve(res);
},
(err) => {
this.urlsScore[i_] = Math.max(URL_SCORE_MIN, this.urlsScore[i_] - URL_SCORE_DELTA_ERROR);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should the score be reduced by half, so that the url will have to prove working multiple times to come back to max

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like currently score decreases twice as fast as score increases

@wemeetagain wemeetagain merged commit 0edf4d6 into unstable Oct 10, 2022
@wemeetagain wemeetagain deleted the dapplion/vc-bn-fallback branch October 10, 2022 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support multiple bn per vc (fallback)
5 participants