Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(beacon-node): network worker new space adjustment #5829

Merged
merged 25 commits into from
Aug 24, 2023

Conversation

matthewkeil
Copy link
Member

@matthewkeil matthewkeil commented Aug 1, 2023

Motivation

While going through the suggestions that Ben gave us I found that there was a very large amount of new space scavenge collection. I updated the dashboards and realized it was over 200mb and 3% of the worker thread time spent on scavenge. There was a lot of page-faults as well which lends itself to a bunch of data being moved during GC. There was also not very much mark-and-sweep GC which means the objects were most likely getting moved only once before getting un-ref'd. This realization that most of the objects are very short lived, but likely high in volume to trigger the scavenge. So in essence causing them to get moved back and forth between the To and From spaces a single time.

I played around with the values of new space to get around this. I bumped the new space way up above the GC volume. Now the objects do not live long enough to need to be moved in the new space. The HandleScope and new space management system are quite efficient so there is little to no performance degradation from leaving the objects in new space once they are no longer ref'd. They just either sit at the back of the line or get written over when the HandleScope pointers shift.

Description

  • Updated dashboard panels for memory.
  • Added panels for each thread (heap and GC stats) in a row
  • Updated network worker new space to a sufficiently large number (over the existing unstable GC volume)

[X] - Figure out method to determine how large is too large
[X] - Verify that new space is not too large

Helps with #5604

** Metrics **

Need to let the node stabilize for a few more hours so traffic is back up to levels on unstable for even comparison but things are looking pretty good so far.

Screenshot 2023-08-01 at 3 55 53 AM
Screenshot 2023-08-01 at 3 56 19 AM
Screenshot 2023-08-01 at 3 56 26 AM
Screenshot 2023-08-01 at 3 56 39 AM

Screenshot 2023-08-01 at 4 00 52 AM
Screenshot 2023-08-01 at 4 00 27 AM
Screenshot 2023-08-01 at 3 57 08 AM

Screenshot 2023-08-01 at 4 02 04 AM
Screenshot 2023-08-01 at 4 02 30 AM
Screenshot 2023-08-01 at 4 01 44 AM
Screenshot 2023-08-01 at 4 04 39 AM
Screenshot 2023-08-01 at 4 04 06 AM
Screenshot 2023-08-01 at 4 03 46 AM
Screenshot 2023-08-01 at 4 02 45 AM

@github-actions
Copy link
Contributor

github-actions bot commented Aug 1, 2023

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 415f7ad Previous: 945f892 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 558.69 us/op 556.88 us/op 1.00
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 76.611 us/op 90.601 us/op 0.85
BLS verify - blst-native 1.2985 ms/op 1.3938 ms/op 0.93
BLS verifyMultipleSignatures 3 - blst-native 2.7238 ms/op 2.8585 ms/op 0.95
BLS verifyMultipleSignatures 8 - blst-native 6.0033 ms/op 6.4062 ms/op 0.94
BLS verifyMultipleSignatures 32 - blst-native 21.915 ms/op 23.454 ms/op 0.93
BLS verifyMultipleSignatures 64 - blst-native 43.020 ms/op 46.889 ms/op 0.92
BLS verifyMultipleSignatures 128 - blst-native 86.485 ms/op 90.563 ms/op 0.95
BLS verifyMultipleSignatures - same message - 3 - blst-native 1.3198 ms/op 1.3891 ms/op 0.95
BLS verifyMultipleSignatures - same message - 8 - blst-native 1.4787 ms/op 1.6709 ms/op 0.88
BLS verifyMultipleSignatures - same message - 32 - blst-native 2.2716 ms/op 2.3986 ms/op 0.95
BLS verifyMultipleSignatures - same message - 64 - blst-native 3.3346 ms/op 3.5379 ms/op 0.94
BLS verifyMultipleSignatures - same message - 128 - blst-native 5.5054 ms/op 5.8087 ms/op 0.95
BLS aggregatePubkeys 32 - blst-native 25.151 us/op 27.418 us/op 0.92
BLS aggregatePubkeys 128 - blst-native 98.811 us/op 108.24 us/op 0.91
getAttestationsForBlock 36.817 ms/op 37.227 ms/op 0.99
isKnown best case - 1 super set check 309.00 ns/op 427.00 ns/op 0.72
isKnown normal case - 2 super set checks 302.00 ns/op 524.00 ns/op 0.58
isKnown worse case - 16 super set checks 280.00 ns/op 447.00 ns/op 0.63
CheckpointStateCache - add get delete 5.4280 us/op 6.1440 us/op 0.88
validate api signedAggregateAndProof - struct 2.7230 ms/op 2.9226 ms/op 0.93
validate gossip signedAggregateAndProof - struct 2.7537 ms/op 2.9227 ms/op 0.94
validate gossip attestation - vc 640000 1.3249 ms/op 1.3975 ms/op 0.95
batch validate gossip attestation - vc 640000 - chunk 32 149.61 us/op 154.91 us/op 0.97
batch validate gossip attestation - vc 640000 - chunk 64 128.11 us/op 137.07 us/op 0.93
batch validate gossip attestation - vc 640000 - chunk 128 115.91 us/op 131.43 us/op 0.88
batch validate gossip attestation - vc 640000 - chunk 256 111.20 us/op 121.26 us/op 0.92
pickEth1Vote - no votes 1.1544 ms/op 1.2926 ms/op 0.89
pickEth1Vote - max votes 10.004 ms/op 11.140 ms/op 0.90
pickEth1Vote - Eth1Data hashTreeRoot value x2048 8.3192 ms/op 9.7212 ms/op 0.86
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 12.964 ms/op 17.008 ms/op 0.76
pickEth1Vote - Eth1Data fastSerialize value x2048 619.89 us/op 649.67 us/op 0.95
pickEth1Vote - Eth1Data fastSerialize tree x2048 5.5913 ms/op 4.9564 ms/op 1.13
bytes32 toHexString 452.00 ns/op 530.00 ns/op 0.85
bytes32 Buffer.toString(hex) 285.00 ns/op 291.00 ns/op 0.98
bytes32 Buffer.toString(hex) from Uint8Array 411.00 ns/op 501.00 ns/op 0.82
bytes32 Buffer.toString(hex) + 0x 288.00 ns/op 299.00 ns/op 0.96
Object access 1 prop 0.14300 ns/op 0.16800 ns/op 0.85
Map access 1 prop 0.14500 ns/op 0.14900 ns/op 0.97
Object get x1000 7.5970 ns/op 8.1450 ns/op 0.93
Map get x1000 0.59400 ns/op 0.58700 ns/op 1.01
Object set x1000 46.845 ns/op 57.508 ns/op 0.81
Map set x1000 38.930 ns/op 44.336 ns/op 0.88
Return object 10000 times 0.22800 ns/op 0.25150 ns/op 0.91
Throw Error 10000 times 3.7987 us/op 4.0456 us/op 0.94
fastMsgIdFn sha256 / 200 bytes 3.2230 us/op 3.4000 us/op 0.95
fastMsgIdFn h32 xxhash / 200 bytes 269.00 ns/op 312.00 ns/op 0.86
fastMsgIdFn h64 xxhash / 200 bytes 343.00 ns/op 383.00 ns/op 0.90
fastMsgIdFn sha256 / 1000 bytes 11.151 us/op 11.708 us/op 0.95
fastMsgIdFn h32 xxhash / 1000 bytes 388.00 ns/op 458.00 ns/op 0.85
fastMsgIdFn h64 xxhash / 1000 bytes 406.00 ns/op 455.00 ns/op 0.89
fastMsgIdFn sha256 / 10000 bytes 101.90 us/op 107.61 us/op 0.95
fastMsgIdFn h32 xxhash / 10000 bytes 1.8930 us/op 2.0070 us/op 0.94
fastMsgIdFn h64 xxhash / 10000 bytes 1.3140 us/op 1.3860 us/op 0.95
enrSubnets - fastDeserialize 64 bits 1.1510 us/op 1.3600 us/op 0.85
enrSubnets - ssz BitVector 64 bits 393.00 ns/op 509.00 ns/op 0.77
enrSubnets - fastDeserialize 4 bits 157.00 ns/op 204.00 ns/op 0.77
enrSubnets - ssz BitVector 4 bits 410.00 ns/op 505.00 ns/op 0.81
prioritizePeers score -10:0 att 32-0.1 sync 2-0 94.033 us/op 116.21 us/op 0.81
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 119.17 us/op 134.05 us/op 0.89
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 156.16 us/op 175.65 us/op 0.89
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 280.16 us/op 319.92 us/op 0.88
prioritizePeers score 0:0 att 64-1 sync 4-1 332.64 us/op 373.64 us/op 0.89
array of 16000 items push then shift 1.6002 us/op 1.6821 us/op 0.95
LinkedList of 16000 items push then shift 8.8310 ns/op 9.6260 ns/op 0.92
array of 16000 items push then pop 80.771 ns/op 97.691 ns/op 0.83
LinkedList of 16000 items push then pop 8.3200 ns/op 9.2470 ns/op 0.90
array of 24000 items push then shift 2.4302 us/op 2.6425 us/op 0.92
LinkedList of 24000 items push then shift 8.8400 ns/op 9.8540 ns/op 0.90
array of 24000 items push then pop 111.99 ns/op 134.52 ns/op 0.83
LinkedList of 24000 items push then pop 8.6220 ns/op 9.2470 ns/op 0.93
intersect bitArray bitLen 8 6.6940 ns/op 7.0320 ns/op 0.95
intersect array and set length 8 53.181 ns/op 77.395 ns/op 0.69
intersect bitArray bitLen 128 31.316 ns/op 32.982 ns/op 0.95
intersect array and set length 128 735.78 ns/op 947.72 ns/op 0.78
bitArray.getTrueBitIndexes() bitLen 128 1.4220 us/op 1.8550 us/op 0.77
bitArray.getTrueBitIndexes() bitLen 248 2.3990 us/op 3.1440 us/op 0.76
bitArray.getTrueBitIndexes() bitLen 512 4.6240 us/op 5.8810 us/op 0.79
Buffer.concat 32 items 928.00 ns/op 1.0830 us/op 0.86
Uint8Array.set 32 items 1.8580 us/op 1.8530 us/op 1.00
Set add up to 64 items then delete first 4.2122 us/op 4.6587 us/op 0.90
OrderedSet add up to 64 items then delete first 5.2468 us/op 5.5648 us/op 0.94
Set add up to 64 items then delete last 4.3945 us/op 4.7629 us/op 0.92
OrderedSet add up to 64 items then delete last 5.6008 us/op 6.0476 us/op 0.93
Set add up to 64 items then delete middle 4.5618 us/op 5.0657 us/op 0.90
OrderedSet add up to 64 items then delete middle 6.6293 us/op 7.4743 us/op 0.89
Set add up to 128 items then delete first 9.2790 us/op 10.415 us/op 0.89
OrderedSet add up to 128 items then delete first 11.844 us/op 13.332 us/op 0.89
Set add up to 128 items then delete last 8.9523 us/op 10.264 us/op 0.87
OrderedSet add up to 128 items then delete last 11.351 us/op 13.302 us/op 0.85
Set add up to 128 items then delete middle 8.6896 us/op 10.158 us/op 0.86
OrderedSet add up to 128 items then delete middle 16.670 us/op 18.991 us/op 0.88
Set add up to 256 items then delete first 18.725 us/op 21.497 us/op 0.87
OrderedSet add up to 256 items then delete first 24.770 us/op 28.766 us/op 0.86
Set add up to 256 items then delete last 17.923 us/op 20.147 us/op 0.89
OrderedSet add up to 256 items then delete last 22.596 us/op 26.729 us/op 0.85
Set add up to 256 items then delete middle 18.009 us/op 20.297 us/op 0.89
OrderedSet add up to 256 items then delete middle 44.875 us/op 49.730 us/op 0.90
transfer serialized Status (84 B) 1.7480 us/op 1.9280 us/op 0.91
copy serialized Status (84 B) 1.5510 us/op 1.6740 us/op 0.93
transfer serialized SignedVoluntaryExit (112 B) 2.0330 us/op 2.1210 us/op 0.96
copy serialized SignedVoluntaryExit (112 B) 1.6770 us/op 1.7290 us/op 0.97
transfer serialized ProposerSlashing (416 B) 3.1820 us/op 2.2110 us/op 1.44
copy serialized ProposerSlashing (416 B) 3.0400 us/op 2.4500 us/op 1.24
transfer serialized Attestation (485 B) 3.2560 us/op 2.8080 us/op 1.16
copy serialized Attestation (485 B) 3.0090 us/op 2.9270 us/op 1.03
transfer serialized AttesterSlashing (33232 B) 3.1940 us/op 2.9350 us/op 1.09
copy serialized AttesterSlashing (33232 B) 5.7140 us/op 7.6820 us/op 0.74
transfer serialized Small SignedBeaconBlock (128000 B) 2.7440 us/op 2.5720 us/op 1.07
copy serialized Small SignedBeaconBlock (128000 B) 14.058 us/op 18.689 us/op 0.75
transfer serialized Avg SignedBeaconBlock (200000 B) 3.1150 us/op 2.9930 us/op 1.04
copy serialized Avg SignedBeaconBlock (200000 B) 17.873 us/op 28.232 us/op 0.63
transfer serialized BlobsSidecar (524380 B) 2.8270 us/op 3.4540 us/op 0.82
copy serialized BlobsSidecar (524380 B) 85.282 us/op 117.41 us/op 0.73
transfer serialized Big SignedBeaconBlock (1000000 B) 2.9480 us/op 3.2340 us/op 0.91
copy serialized Big SignedBeaconBlock (1000000 B) 162.38 us/op 451.05 us/op 0.36
pass gossip attestations to forkchoice per slot 3.7473 ms/op 3.9944 ms/op 0.94
forkChoice updateHead vc 100000 bc 64 eq 0 660.13 us/op 700.99 us/op 0.94
forkChoice updateHead vc 600000 bc 64 eq 0 4.0397 ms/op 5.8807 ms/op 0.69
forkChoice updateHead vc 1000000 bc 64 eq 0 7.2375 ms/op 9.2932 ms/op 0.78
forkChoice updateHead vc 600000 bc 320 eq 0 4.1849 ms/op 4.5954 ms/op 0.91
forkChoice updateHead vc 600000 bc 1200 eq 0 4.1458 ms/op 4.6986 ms/op 0.88
forkChoice updateHead vc 600000 bc 7200 eq 0 5.1073 ms/op 5.5087 ms/op 0.93
forkChoice updateHead vc 600000 bc 64 eq 1000 11.196 ms/op 11.687 ms/op 0.96
forkChoice updateHead vc 600000 bc 64 eq 10000 12.055 ms/op 12.697 ms/op 0.95
forkChoice updateHead vc 600000 bc 64 eq 300000 15.859 ms/op 18.504 ms/op 0.86
computeDeltas 500000 validators 300 proto nodes 6.2419 ms/op 6.6052 ms/op 0.94
computeDeltas 500000 validators 1200 proto nodes 6.1674 ms/op 6.6995 ms/op 0.92
computeDeltas 500000 validators 7200 proto nodes 6.0053 ms/op 6.8328 ms/op 0.88
computeDeltas 750000 validators 300 proto nodes 9.0900 ms/op 10.150 ms/op 0.90
computeDeltas 750000 validators 1200 proto nodes 9.1425 ms/op 10.207 ms/op 0.90
computeDeltas 750000 validators 7200 proto nodes 9.1931 ms/op 10.534 ms/op 0.87
computeDeltas 1400000 validators 300 proto nodes 17.430 ms/op 19.124 ms/op 0.91
computeDeltas 1400000 validators 1200 proto nodes 17.303 ms/op 18.508 ms/op 0.93
computeDeltas 1400000 validators 7200 proto nodes 17.163 ms/op 18.569 ms/op 0.92
computeDeltas 2100000 validators 300 proto nodes 25.713 ms/op 27.486 ms/op 0.94
computeDeltas 2100000 validators 1200 proto nodes 26.030 ms/op 28.393 ms/op 0.92
computeDeltas 2100000 validators 7200 proto nodes 26.001 ms/op 27.848 ms/op 0.93
computeProposerBoostScoreFromBalances 500000 validators 3.1421 ms/op 3.3261 ms/op 0.94
computeProposerBoostScoreFromBalances 750000 validators 3.1855 ms/op 3.3978 ms/op 0.94
computeProposerBoostScoreFromBalances 1400000 validators 3.2162 ms/op 3.3742 ms/op 0.95
computeProposerBoostScoreFromBalances 2100000 validators 3.2144 ms/op 3.4240 ms/op 0.94
altair processAttestation - 250000 vs - 7PWei normalcase 2.4409 ms/op 3.3512 ms/op 0.73
altair processAttestation - 250000 vs - 7PWei worstcase 3.3653 ms/op 4.1484 ms/op 0.81
altair processAttestation - setStatus - 1/6 committees join 183.91 us/op 187.58 us/op 0.98
altair processAttestation - setStatus - 1/3 committees join 349.13 us/op 354.18 us/op 0.99
altair processAttestation - setStatus - 1/2 committees join 470.01 us/op 480.52 us/op 0.98
altair processAttestation - setStatus - 2/3 committees join 597.09 us/op 634.13 us/op 0.94
altair processAttestation - setStatus - 4/5 committees join 808.23 us/op 810.11 us/op 1.00
altair processAttestation - setStatus - 100% committees join 949.82 us/op 969.14 us/op 0.98
altair processBlock - 250000 vs - 7PWei normalcase 9.2076 ms/op 9.9540 ms/op 0.93
altair processBlock - 250000 vs - 7PWei normalcase hashState 17.913 ms/op 17.957 ms/op 1.00
altair processBlock - 250000 vs - 7PWei worstcase 38.433 ms/op 39.884 ms/op 0.96
altair processBlock - 250000 vs - 7PWei worstcase hashState 58.558 ms/op 65.361 ms/op 0.90
phase0 processBlock - 250000 vs - 7PWei normalcase 2.3670 ms/op 2.7308 ms/op 0.87
phase0 processBlock - 250000 vs - 7PWei worstcase 29.581 ms/op 33.560 ms/op 0.88
altair processEth1Data - 250000 vs - 7PWei normalcase 470.10 us/op 561.91 us/op 0.84
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 12.628 us/op 11.141 us/op 1.13
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 56.726 us/op 44.504 us/op 1.27
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 10.821 us/op 22.985 us/op 0.47
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 7.8110 us/op 16.176 us/op 0.48
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 139.41 us/op 150.26 us/op 0.93
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.0028 ms/op 1.1883 ms/op 0.84
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.4068 ms/op 1.8035 ms/op 0.78
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.8798 ms/op 1.7205 ms/op 1.09
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.2734 ms/op 3.5696 ms/op 0.92
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 2.0761 ms/op 2.4484 ms/op 0.85
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 4.3993 ms/op 5.1072 ms/op 0.86
Tree 40 250000 create 303.71 ms/op 331.65 ms/op 0.92
Tree 40 250000 get(125000) 191.59 ns/op 207.60 ns/op 0.92
Tree 40 250000 set(125000) 863.88 ns/op 971.17 ns/op 0.89
Tree 40 250000 toArray() 17.678 ms/op 19.075 ms/op 0.93
Tree 40 250000 iterate all - toArray() + loop 17.737 ms/op 18.257 ms/op 0.97
Tree 40 250000 iterate all - get(i) 64.401 ms/op 68.658 ms/op 0.94
MutableVector 250000 create 11.999 ms/op 13.301 ms/op 0.90
MutableVector 250000 get(125000) 6.2310 ns/op 6.8390 ns/op 0.91
MutableVector 250000 set(125000) 253.95 ns/op 270.19 ns/op 0.94
MutableVector 250000 toArray() 3.5057 ms/op 3.3718 ms/op 1.04
MutableVector 250000 iterate all - toArray() + loop 3.0609 ms/op 3.3196 ms/op 0.92
MutableVector 250000 iterate all - get(i) 1.5352 ms/op 1.5662 ms/op 0.98
Array 250000 create 2.6199 ms/op 2.9943 ms/op 0.87
Array 250000 clone - spread 978.32 us/op 1.1939 ms/op 0.82
Array 250000 get(125000) 0.48600 ns/op 0.59500 ns/op 0.82
Array 250000 set(125000) 0.56800 ns/op 0.66200 ns/op 0.86
Array 250000 iterate all - loop 81.003 us/op 109.76 us/op 0.74
effectiveBalanceIncrements clone Uint8Array 300000 22.087 us/op 29.293 us/op 0.75
effectiveBalanceIncrements clone MutableVector 300000 260.00 ns/op 355.00 ns/op 0.73
effectiveBalanceIncrements rw all Uint8Array 300000 175.55 us/op 181.14 us/op 0.97
effectiveBalanceIncrements rw all MutableVector 300000 73.865 ms/op 89.352 ms/op 0.83
phase0 afterProcessEpoch - 250000 vs - 7PWei 110.51 ms/op 117.74 ms/op 0.94
phase0 beforeProcessEpoch - 250000 vs - 7PWei 37.522 ms/op 33.198 ms/op 1.13
altair processEpoch - mainnet_e81889 306.41 ms/op 327.84 ms/op 0.93
mainnet_e81889 - altair beforeProcessEpoch 46.900 ms/op 47.923 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 16.533 us/op 19.062 us/op 0.87
mainnet_e81889 - altair processInactivityUpdates 5.0800 ms/op 6.3084 ms/op 0.81
mainnet_e81889 - altair processRewardsAndPenalties 51.492 ms/op 67.876 ms/op 0.76
mainnet_e81889 - altair processRegistryUpdates 2.2270 us/op 2.2140 us/op 1.01
mainnet_e81889 - altair processSlashings 411.00 ns/op 423.00 ns/op 0.97
mainnet_e81889 - altair processEth1DataReset 520.00 ns/op 525.00 ns/op 0.99
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.2454 ms/op 1.3065 ms/op 0.95
mainnet_e81889 - altair processSlashingsReset 2.8700 us/op 3.0460 us/op 0.94
mainnet_e81889 - altair processRandaoMixesReset 3.9930 us/op 4.4270 us/op 0.90
mainnet_e81889 - altair processHistoricalRootsUpdate 1.0850 us/op 731.00 ns/op 1.48
mainnet_e81889 - altair processParticipationFlagUpdates 1.9040 us/op 1.6490 us/op 1.15
mainnet_e81889 - altair processSyncCommitteeUpdates 734.00 ns/op 938.00 ns/op 0.78
mainnet_e81889 - altair afterProcessEpoch 121.82 ms/op 129.08 ms/op 0.94
capella processEpoch - mainnet_e217614 1.0352 s/op 1.0143 s/op 1.02
mainnet_e217614 - capella beforeProcessEpoch 230.06 ms/op 224.84 ms/op 1.02
mainnet_e217614 - capella processJustificationAndFinalization 13.182 us/op 13.566 us/op 0.97
mainnet_e217614 - capella processInactivityUpdates 19.673 ms/op 18.256 ms/op 1.08
mainnet_e217614 - capella processRewardsAndPenalties 279.06 ms/op 286.83 ms/op 0.97
mainnet_e217614 - capella processRegistryUpdates 19.566 us/op 18.247 us/op 1.07
mainnet_e217614 - capella processSlashings 518.00 ns/op 511.00 ns/op 1.01
mainnet_e217614 - capella processEth1DataReset 543.00 ns/op 454.00 ns/op 1.20
mainnet_e217614 - capella processEffectiveBalanceUpdates 4.0383 ms/op 4.2756 ms/op 0.94
mainnet_e217614 - capella processSlashingsReset 2.3450 us/op 2.2020 us/op 1.06
mainnet_e217614 - capella processRandaoMixesReset 4.4020 us/op 4.6150 us/op 0.95
mainnet_e217614 - capella processHistoricalRootsUpdate 664.00 ns/op 482.00 ns/op 1.38
mainnet_e217614 - capella processParticipationFlagUpdates 4.9710 us/op 2.8770 us/op 1.73
mainnet_e217614 - capella afterProcessEpoch 299.24 ms/op 305.30 ms/op 0.98
phase0 processEpoch - mainnet_e58758 327.64 ms/op 331.89 ms/op 0.99
mainnet_e58758 - phase0 beforeProcessEpoch 112.88 ms/op 103.58 ms/op 1.09
mainnet_e58758 - phase0 processJustificationAndFinalization 14.440 us/op 15.992 us/op 0.90
mainnet_e58758 - phase0 processRewardsAndPenalties 44.682 ms/op 51.638 ms/op 0.87
mainnet_e58758 - phase0 processRegistryUpdates 11.185 us/op 14.024 us/op 0.80
mainnet_e58758 - phase0 processSlashings 538.00 ns/op 529.00 ns/op 1.02
mainnet_e58758 - phase0 processEth1DataReset 473.00 ns/op 621.00 ns/op 0.76
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.0401 ms/op 1.1513 ms/op 0.90
mainnet_e58758 - phase0 processSlashingsReset 2.3110 us/op 2.7650 us/op 0.84
mainnet_e58758 - phase0 processRandaoMixesReset 4.2240 us/op 6.0150 us/op 0.70
mainnet_e58758 - phase0 processHistoricalRootsUpdate 638.00 ns/op 908.00 ns/op 0.70
mainnet_e58758 - phase0 processParticipationRecordUpdates 6.0010 us/op 6.5620 us/op 0.91
mainnet_e58758 - phase0 afterProcessEpoch 103.17 ms/op 106.57 ms/op 0.97
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9416 ms/op 1.3547 ms/op 1.43
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.4378 ms/op 1.8201 ms/op 0.79
altair processInactivityUpdates - 250000 normalcase 25.875 ms/op 28.594 ms/op 0.90
altair processInactivityUpdates - 250000 worstcase 26.468 ms/op 28.687 ms/op 0.92
phase0 processRegistryUpdates - 250000 normalcase 9.7960 us/op 12.363 us/op 0.79
phase0 processRegistryUpdates - 250000 badcase_full_deposits 373.02 us/op 382.94 us/op 0.97
phase0 processRegistryUpdates - 250000 worstcase 0.5 140.26 ms/op 125.34 ms/op 1.12
altair processRewardsAndPenalties - 250000 normalcase 74.855 ms/op 73.085 ms/op 1.02
altair processRewardsAndPenalties - 250000 worstcase 75.122 ms/op 70.814 ms/op 1.06
phase0 getAttestationDeltas - 250000 normalcase 8.5880 ms/op 8.7973 ms/op 0.98
phase0 getAttestationDeltas - 250000 worstcase 7.5986 ms/op 9.7656 ms/op 0.78
phase0 processSlashings - 250000 worstcase 2.2374 ms/op 2.5356 ms/op 0.88
altair processSyncCommitteeUpdates - 250000 143.37 ms/op 162.80 ms/op 0.88
BeaconState.hashTreeRoot - No change 269.00 ns/op 270.00 ns/op 1.00
BeaconState.hashTreeRoot - 1 full validator 51.858 us/op 50.056 us/op 1.04
BeaconState.hashTreeRoot - 32 full validator 519.87 us/op 552.19 us/op 0.94
BeaconState.hashTreeRoot - 512 full validator 5.6030 ms/op 6.1991 ms/op 0.90
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 61.507 us/op 64.905 us/op 0.95
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 844.87 us/op 977.53 us/op 0.86
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 11.000 ms/op 12.146 ms/op 0.91
BeaconState.hashTreeRoot - 1 balances 49.553 us/op 52.515 us/op 0.94
BeaconState.hashTreeRoot - 32 balances 423.50 us/op 485.40 us/op 0.87
BeaconState.hashTreeRoot - 512 balances 4.1761 ms/op 4.8128 ms/op 0.87
BeaconState.hashTreeRoot - 250000 balances 76.187 ms/op 73.162 ms/op 1.04
aggregationBits - 2048 els - zipIndexesInBitList 15.308 us/op 17.668 us/op 0.87
regular array get 100000 times 32.535 us/op 40.999 us/op 0.79
wrappedArray get 100000 times 31.669 us/op 34.154 us/op 0.93
arrayWithProxy get 100000 times 13.741 ms/op 15.219 ms/op 0.90
ssz.Root.equals 210.00 ns/op 245.00 ns/op 0.86
byteArrayEquals 210.00 ns/op 256.00 ns/op 0.82
shuffle list - 16384 els 6.9179 ms/op 7.4460 ms/op 0.93
shuffle list - 250000 els 100.45 ms/op 107.32 ms/op 0.94
processSlot - 1 slots 8.2450 us/op 10.519 us/op 0.78
processSlot - 32 slots 1.3137 ms/op 1.4672 ms/op 0.90
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 54.579 ms/op 57.007 ms/op 0.96
getCommitteeAssignments - req 1 vs - 250000 vc 2.5473 ms/op 2.6560 ms/op 0.96
getCommitteeAssignments - req 100 vs - 250000 vc 3.7433 ms/op 3.8518 ms/op 0.97
getCommitteeAssignments - req 1000 vs - 250000 vc 4.1223 ms/op 4.2418 ms/op 0.97
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.9800 ns/op 5.6100 ns/op 0.89
state getBlockRootAtSlot - 250000 vs - 7PWei 694.98 ns/op 769.22 ns/op 0.90
computeProposers - vc 250000 10.048 ms/op 10.013 ms/op 1.00
computeEpochShuffling - vc 250000 108.34 ms/op 111.77 ms/op 0.97
getNextSyncCommittee - vc 250000 153.79 ms/op 165.73 ms/op 0.93
computeSigningRoot for AttestationData 12.868 us/op 14.361 us/op 0.90
hash AttestationData serialized data then Buffer.toString(base64) 2.3676 us/op 2.4072 us/op 0.98
toHexString serialized data 1.1949 us/op 1.0978 us/op 1.09
Buffer.toString(base64) 218.77 ns/op 231.27 ns/op 0.95

by benchmarkbot/action

@dapplion
Copy link
Contributor

dapplion commented Aug 3, 2023

Metrics looks promising! This PR does not add metrics, but changes a fundamental parameter, can you split:

  • PR A: Add dashboards
  • PR B: Change constant value

@matthewkeil matthewkeil force-pushed the mkeil/network-worker-new-space branch from 49fc485 to ed0b59c Compare August 5, 2023 03:24
@matthewkeil
Copy link
Member Author

matthewkeil commented Aug 7, 2023

This was updated to just have the single line change. I rebased this onto unstable from a discussion with @wemeetagain but there is a memory leak on unstable so its hard to see how this is actually performing at 128mb. I will move to ready for review once we get the leak sorted and can see how this performs with @tuyennhv update with limited subscriptions.

Dashboard changes were moved to PR #5858

@matthewkeil
Copy link
Member Author

matthewkeil commented Aug 22, 2023

Value was set from research on #2115. You can find the resolution to that issue here

There is a 3x1 ratio of semi space to young generation space in the code but in practice it seems like setting the semi space to the average value of scavenge collection seemed optimal. In previous empirical testing it seemed to also be the optimal value for the worker young generation setting despite the ratio. I am not certain of the reason for this but its worth further investigation.

@matthewkeil matthewkeil marked this pull request as ready for review August 22, 2023 10:08
@matthewkeil matthewkeil requested a review from a team as a code owner August 22, 2023 10:08
@philknows philknows added this to the v1.11.0 milestone Aug 22, 2023
wemeetagain
wemeetagain previously approved these changes Aug 23, 2023
@wemeetagain wemeetagain enabled auto-merge (squash) August 23, 2023 22:26
@wemeetagain wemeetagain disabled auto-merge August 24, 2023 22:32
@wemeetagain wemeetagain merged commit 1b40a91 into unstable Aug 24, 2023
@wemeetagain wemeetagain deleted the mkeil/network-worker-new-space branch August 24, 2023 22:33
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.11.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants