Increase committee size in sim tests

[twoeths]

Motivation

Increase committee size in sim tests to avoid any regressions we may have

[codecov]

Codecov Report

Merging #3921 (5fe1338) into master (053f76a) will not change coverage.
The diff coverage is n/a.

❗ Current head 5fe1338 differs from pull request most recent head 1cc5197. Consider uploading reports for the commit 1cc5197 to get more accurate results

@@           Coverage Diff           @@
##           master    #3921   +/-   ##
=======================================
  Coverage   36.32%   36.32%           
=======================================
  Files         324      324           
  Lines        9059     9059           
  Branches     1453     1453           
=======================================
  Hits         3291     3291           
  Misses       5597     5597           
  Partials      171      171           

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 5fe1338	Previous: 053f76a	Ratio
BeaconState.hashTreeRoot - No change	503.00 ns/op	535.00 ns/op	0.94
BeaconState.hashTreeRoot - 1 full validator	50.758 us/op	68.699 us/op	0.74
BeaconState.hashTreeRoot - 32 full validator	503.24 us/op	676.18 us/op	0.74
BeaconState.hashTreeRoot - 512 full validator	5.6472 ms/op	7.3856 ms/op	0.76
BeaconState.hashTreeRoot - 1 validator.effectiveBalance	61.899 us/op	84.960 us/op	0.73
BeaconState.hashTreeRoot - 32 validator.effectiveBalance	907.12 us/op	1.1821 ms/op	0.77
BeaconState.hashTreeRoot - 512 validator.effectiveBalance	11.972 ms/op	16.455 ms/op	0.73
BeaconState.hashTreeRoot - 1 balances	47.677 us/op	66.190 us/op	0.72
BeaconState.hashTreeRoot - 32 balances	436.86 us/op	579.14 us/op	0.75
BeaconState.hashTreeRoot - 512 balances	4.2420 ms/op	5.6276 ms/op	0.75
BeaconState.hashTreeRoot - 250000 balances	92.505 ms/op	125.16 ms/op	0.74
processSlot - 1 slots	8.8150 us/op	12.122 us/op	0.73
processSlot - 32 slots	1.4020 ms/op	1.9337 ms/op	0.73
getCommitteeAssignments - req 1 vs - 250000 vc	4.8137 ms/op	6.5325 ms/op	0.74
getCommitteeAssignments - req 100 vs - 250000 vc	6.7616 ms/op	9.0426 ms/op	0.75
getCommitteeAssignments - req 1000 vs - 250000 vc	7.4160 ms/op	9.7459 ms/op	0.76
computeProposers - vc 250000	15.625 ms/op	20.387 ms/op	0.77
computeEpochShuffling - vc 250000	149.43 ms/op	196.37 ms/op	0.76
getNextSyncCommittee - vc 250000	256.59 ms/op	337.43 ms/op	0.76
altair processAttestation - 250000 vs - 7PWei normalcase	3.2819 ms/op	4.7086 ms/op	0.70
altair processAttestation - 250000 vs - 7PWei worstcase	5.0968 ms/op	6.6570 ms/op	0.77
altair processAttestation - setStatus - 1/6 committees join	177.78 us/op	236.08 us/op	0.75
altair processAttestation - setStatus - 1/3 committees join	331.13 us/op	457.64 us/op	0.72
altair processAttestation - setStatus - 1/2 committees join	481.01 us/op	644.50 us/op	0.75
altair processAttestation - setStatus - 2/3 committees join	636.12 us/op	833.75 us/op	0.76
altair processAttestation - setStatus - 4/5 committees join	896.94 us/op	1.1570 ms/op	0.78
altair processAttestation - setStatus - 100% committees join	1.0379 ms/op	1.3652 ms/op	0.76
altair processBlock - 250000 vs - 7PWei normalcase	22.768 ms/op	32.723 ms/op	0.70
altair processBlock - 250000 vs - 7PWei normalcase hashState	32.103 ms/op	41.797 ms/op	0.77
altair processBlock - 250000 vs - 7PWei worstcase	77.976 ms/op	86.428 ms/op	0.90
altair processBlock - 250000 vs - 7PWei worstcase hashState	107.44 ms/op	130.46 ms/op	0.82
altair processEth1Data - 250000 vs - 7PWei normalcase	738.61 us/op	926.92 us/op	0.80
altair processEpoch - mainnet_e81889	501.47 ms/op	643.84 ms/op	0.78
mainnet_e81889 - altair beforeProcessEpoch	138.12 ms/op	164.56 ms/op	0.84
mainnet_e81889 - altair processJustificationAndFinalization	54.692 us/op	36.219 us/op	1.51
mainnet_e81889 - altair processInactivityUpdates	9.8525 ms/op	12.251 ms/op	0.80
mainnet_e81889 - altair processRewardsAndPenalties	81.803 ms/op	98.826 ms/op	0.83
mainnet_e81889 - altair processRegistryUpdates	12.118 us/op	7.9700 us/op	1.52
mainnet_e81889 - altair processSlashings	4.8630 us/op	3.4180 us/op	1.42
mainnet_e81889 - altair processEth1DataReset	4.6820 us/op	3.8170 us/op	1.23
mainnet_e81889 - altair processEffectiveBalanceUpdates	6.5946 ms/op	7.8304 ms/op	0.84
mainnet_e81889 - altair processSlashingsReset	17.084 us/op	10.600 us/op	1.61
mainnet_e81889 - altair processRandaoMixesReset	20.291 us/op	14.971 us/op	1.36
mainnet_e81889 - altair processHistoricalRootsUpdate	6.1060 us/op	5.0810 us/op	1.20
mainnet_e81889 - altair processParticipationFlagUpdates	10.732 us/op	8.2480 us/op	1.30
mainnet_e81889 - altair processSyncCommitteeUpdates	3.5980 us/op	3.0600 us/op	1.18
mainnet_e81889 - altair afterProcessEpoch	171.67 ms/op	236.15 ms/op	0.73
altair processInactivityUpdates - 250000 normalcase	30.849 ms/op	38.672 ms/op	0.80
altair processInactivityUpdates - 250000 worstcase	25.457 ms/op	32.996 ms/op	0.77
altair processRewardsAndPenalties - 250000 normalcase	108.06 ms/op	137.70 ms/op	0.78
altair processRewardsAndPenalties - 250000 worstcase	74.308 ms/op	118.95 ms/op	0.62
altair processSyncCommitteeUpdates - 250000	285.45 ms/op	352.76 ms/op	0.81
Tree 40 250000 create	609.94 ms/op	798.25 ms/op	0.76
Tree 40 250000 get(125000)	270.12 ns/op	355.67 ns/op	0.76
Tree 40 250000 set(125000)	1.9648 us/op	2.5095 us/op	0.78
Tree 40 250000 toArray()	27.842 ms/op	36.990 ms/op	0.75
Tree 40 250000 iterate all - toArray() + loop	29.270 ms/op	37.153 ms/op	0.79
Tree 40 250000 iterate all - get(i)	105.72 ms/op	135.90 ms/op	0.78
MutableVector 250000 create	15.814 ms/op	17.990 ms/op	0.88
MutableVector 250000 get(125000)	13.578 ns/op	18.625 ns/op	0.73
MutableVector 250000 set(125000)	510.98 ns/op	674.10 ns/op	0.76
MutableVector 250000 toArray()	5.8152 ms/op	7.6934 ms/op	0.76
MutableVector 250000 iterate all - toArray() + loop	6.0115 ms/op	8.0899 ms/op	0.74
MutableVector 250000 iterate all - get(i)	3.0761 ms/op	4.1039 ms/op	0.75
Array 250000 create	5.4510 ms/op	7.3031 ms/op	0.75
Array 250000 clone - spread	1.6977 ms/op	16.561 ms/op	0.10
Array 250000 get(125000)	0.83800 ns/op	1.2660 ns/op	0.66
Array 250000 set(125000)	0.83900 ns/op	1.1350 ns/op	0.74
Array 250000 iterate all - loop	153.42 us/op	209.80 us/op	0.73
effectiveBalanceIncrements clone Uint8Array 300000	54.346 us/op	78.754 us/op	0.69
effectiveBalanceIncrements clone MutableVector 300000	550.00 ns/op	662.00 ns/op	0.83
effectiveBalanceIncrements rw all Uint8Array 300000	282.39 us/op	377.37 us/op	0.75
effectiveBalanceIncrements rw all MutableVector 300000	133.57 ms/op	186.20 ms/op	0.72
aggregationBits - 2048 els - zipIndexesInBitList	20.864 us/op	31.187 us/op	0.67
regular array get 100000 times	64.890 us/op	84.393 us/op	0.77
wrappedArray get 100000 times	65.568 us/op	84.367 us/op	0.78
arrayWithProxy get 100000 times	31.423 ms/op	40.807 ms/op	0.77
ssz.Root.equals	385.00 ns/op	559.00 ns/op	0.69
byteArrayEquals	355.00 ns/op	536.00 ns/op	0.66
phase0 processBlock - 250000 vs - 7PWei normalcase	3.0200 ms/op	4.2272 ms/op	0.71
phase0 processBlock - 250000 vs - 7PWei worstcase	43.520 ms/op	56.547 ms/op	0.77
phase0 afterProcessEpoch - 250000 vs - 7PWei	168.03 ms/op	215.92 ms/op	0.78
phase0 beforeProcessEpoch - 250000 vs - 7PWei	84.652 ms/op	87.634 ms/op	0.97
phase0 processEpoch - mainnet_e58758	510.19 ms/op	626.35 ms/op	0.81
mainnet_e58758 - phase0 beforeProcessEpoch	201.49 ms/op	238.53 ms/op	0.84
mainnet_e58758 - phase0 processJustificationAndFinalization	52.107 us/op	33.411 us/op	1.56
mainnet_e58758 - phase0 processRewardsAndPenalties	69.483 ms/op	84.712 ms/op	0.82
mainnet_e58758 - phase0 processRegistryUpdates	27.114 us/op	18.356 us/op	1.48
mainnet_e58758 - phase0 processSlashings	5.2750 us/op	3.6970 us/op	1.43
mainnet_e58758 - phase0 processEth1DataReset	4.5560 us/op	3.5710 us/op	1.28
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	5.6484 ms/op	7.0909 ms/op	0.80
mainnet_e58758 - phase0 processSlashingsReset	16.880 us/op	11.942 us/op	1.41
mainnet_e58758 - phase0 processRandaoMixesReset	20.065 us/op	12.553 us/op	1.60
mainnet_e58758 - phase0 processHistoricalRootsUpdate	6.0570 us/op	4.1550 us/op	1.46
mainnet_e58758 - phase0 processParticipationRecordUpdates	15.945 us/op	11.525 us/op	1.38
mainnet_e58758 - phase0 afterProcessEpoch	147.38 ms/op	190.73 ms/op	0.77
phase0 processEffectiveBalanceUpdates - 250000 normalcase	6.2385 ms/op	8.0666 ms/op	0.77
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	6.5881 ms/op	8.5646 ms/op	0.77
phase0 processRegistryUpdates - 250000 normalcase	21.392 us/op	14.969 us/op	1.43
phase0 processRegistryUpdates - 250000 badcase_full_deposits	605.08 us/op	566.89 us/op	1.07
phase0 processRegistryUpdates - 250000 worstcase 0.5	199.03 ms/op	281.31 ms/op	0.71
phase0 getAttestationDeltas - 250000 normalcase	13.458 ms/op	17.621 ms/op	0.76
phase0 getAttestationDeltas - 250000 worstcase	14.198 ms/op	17.888 ms/op	0.79
phase0 processSlashings - 250000 worstcase	5.7917 ms/op	6.9201 ms/op	0.84
shuffle list - 16384 els	10.501 ms/op	13.275 ms/op	0.79
shuffle list - 250000 els	151.27 ms/op	191.74 ms/op	0.79
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei	345.23 us/op	476.39 us/op	0.72
pass gossip attestations to forkchoice per slot	3.3533 ms/op	4.1608 ms/op	0.81
computeDeltas	3.0142 ms/op	3.8443 ms/op	0.78
computeProposerBoostScoreFromBalances	483.98 us/op	629.05 us/op	0.77
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	1.6456 ms/op	2.3390 ms/op	0.70
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	70.710 us/op	93.552 us/op	0.76
BLS verify - blst-native	1.6812 ms/op	2.3228 ms/op	0.72
BLS verifyMultipleSignatures 3 - blst-native	3.5089 ms/op	4.7560 ms/op	0.74
BLS verifyMultipleSignatures 8 - blst-native	7.4144 ms/op	10.249 ms/op	0.72
BLS verifyMultipleSignatures 32 - blst-native	28.060 ms/op	37.111 ms/op	0.76
BLS aggregatePubkeys 32 - blst-native	36.474 us/op	49.548 us/op	0.74
BLS aggregatePubkeys 128 - blst-native	145.48 us/op	192.01 us/op	0.76
getAttestationsForBlock	57.343 ms/op	73.927 ms/op	0.78
CheckpointStateCache - add get delete	9.6040 us/op	12.579 us/op	0.76
validate gossip signedAggregateAndProof - struct	4.1547 ms/op	5.3228 ms/op	0.78
validate gossip attestation - struct	1.9816 ms/op	2.5468 ms/op	0.78
pickEth1Vote - no votes	2.0896 ms/op	2.7270 ms/op	0.77
pickEth1Vote - max votes	17.886 ms/op	22.718 ms/op	0.79
pickEth1Vote - Eth1Data hashTreeRoot value x2048	11.443 ms/op	14.599 ms/op	0.78
pickEth1Vote - Eth1Data hashTreeRoot tree x2048	20.457 ms/op	24.629 ms/op	0.83
pickEth1Vote - Eth1Data fastSerialize value x2048	1.6601 ms/op	2.0945 ms/op	0.79
pickEth1Vote - Eth1Data fastSerialize tree x2048	11.851 ms/op	14.860 ms/op	0.80
bytes32 toHexString	892.00 ns/op	1.2540 us/op	0.71
bytes32 Buffer.toString(hex)	647.00 ns/op	848.00 ns/op	0.76
bytes32 Buffer.toString(hex) from Uint8Array	901.00 ns/op	1.1040 us/op	0.82
bytes32 Buffer.toString(hex) + 0x	645.00 ns/op	853.00 ns/op	0.76
Object access 1 prop	0.28200 ns/op	0.42400 ns/op	0.67
Map access 1 prop	0.26000 ns/op	0.37600 ns/op	0.69
Object get x1000	15.888 ns/op	22.319 ns/op	0.71
Map get x1000	1.0190 ns/op	1.2640 ns/op	0.81
Object set x1000	97.943 ns/op	130.50 ns/op	0.75
Map set x1000	55.914 ns/op	78.629 ns/op	0.71
Return object 10000 times	0.34860 ns/op	0.46460 ns/op	0.75
Throw Error 10000 times	5.7071 us/op	7.2765 us/op	0.78
enrSubnets - fastDeserialize 64 bits	736.00 ns/op	1.1400 us/op	0.65
enrSubnets - ssz BitVector 64 bits	550.00 ns/op	756.00 ns/op	0.73
enrSubnets - fastDeserialize 4 bits	347.00 ns/op	482.00 ns/op	0.72
enrSubnets - ssz BitVector 4 bits	535.00 ns/op	748.00 ns/op	0.72
RateTracker 1000000 limit, 1 obj count per request	161.85 ns/op	222.55 ns/op	0.73
RateTracker 1000000 limit, 2 obj count per request	120.46 ns/op	166.14 ns/op	0.73
RateTracker 1000000 limit, 4 obj count per request	101.01 ns/op	138.09 ns/op	0.73
RateTracker 1000000 limit, 8 obj count per request	91.106 ns/op	123.04 ns/op	0.74
RateTracker with prune	3.2890 us/op	4.4870 us/op	0.73
array of 16000 items push then shift	2.9974 us/op	3.9529 us/op	0.76
LinkedList of 16000 items push then shift	14.870 ns/op	20.680 ns/op	0.72
array of 16000 items push then pop	197.78 ns/op	240.70 ns/op	0.82
LinkedList of 16000 items push then pop	14.828 ns/op	18.636 ns/op	0.80
array of 24000 items push then shift	4.1791 us/op	5.6779 us/op	0.74
LinkedList of 24000 items push then shift	16.511 ns/op	23.551 ns/op	0.70
array of 24000 items push then pop	178.97 ns/op	230.01 ns/op	0.78
LinkedList of 24000 items push then pop	15.615 ns/op	20.128 ns/op	0.78

by benchmarkbot/action

[philknows]

Implemented due to #3919 not being detected from sim tests potentially from small committee sizes. Taking note that we should also seek to improve our sim tests with more depth when able.