Run all ssz_static test automatically

[dapplion]

Motivation

Extends #3017 ssz_static tests are not run entirely, but we provide a manual list of test cases to run

Description

Read all files from disk to make sure all test cases are executed

[codeclimate]

Code Climate has analyzed commit 1ae7d53 and detected 0 issues on this pull request.

View more on Code Climate.

[github-actions]

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: 160170c	Previous: 6e21c7c	Ratio
state hashTreeRoot - No change	809.00 ns/op	901.00 ns/op	0.90
state hashTreeRoot - 1 full validator	859.00 ns/op	864.00 ns/op	0.99
state hashTreeRoot - 32 full validator	800.00 ns/op	1.0980 us/op	0.73
state hashTreeRoot - 512 full validator	832.00 ns/op	877.00 ns/op	0.95
state hashTreeRoot - 1 validator.effectiveBalance	769.00 ns/op	856.00 ns/op	0.90
state hashTreeRoot - 32 validator.effectiveBalance	801.00 ns/op	902.00 ns/op	0.89
state hashTreeRoot - 512 validator.effectiveBalance	772.00 ns/op	873.00 ns/op	0.88
state hashTreeRoot - 1 balances	787.00 ns/op	852.00 ns/op	0.92
state hashTreeRoot - 32 balances	779.00 ns/op	893.00 ns/op	0.87
state hashTreeRoot - 512 balances	760.00 ns/op	882.00 ns/op	0.86
state hashTreeRoot - 250000 balances	799.00 ns/op	919.00 ns/op	0.87
processSlot - 1 slots	76.546 us/op	79.678 us/op	0.96
processSlot - 32 slots	3.5544 ms/op	3.8338 ms/op	0.93
getCommitteeAssignments - req 1 vs - 250000 vc	5.5139 ms/op	6.0874 ms/op	0.91
getCommitteeAssignments - req 100 vs - 250000 vc	8.1044 ms/op	8.5631 ms/op	0.95
getCommitteeAssignments - req 1000 vs - 250000 vc	8.6354 ms/op	9.1707 ms/op	0.94
altair processBlock - 250000 vs - 7PWei normalcase	113.83 ms/op	125.09 ms/op	0.91
altair processBlock - 250000 vs - 7PWei worstcase	303.63 ms/op	323.18 ms/op	0.94
altair processEpoch - pyrmont_e62330	806.97 ms/op	846.13 ms/op	0.95
pyrmont_e62330 - altair beforeProcessEpoch	76.522 ms/op	83.532 ms/op	0.92
pyrmont_e62330 - altair processJustificationAndFinalization	154.57 us/op	163.37 us/op	0.95
pyrmont_e62330 - altair processInactivityUpdates	82.661 ms/op	83.568 ms/op	0.99
pyrmont_e62330 - altair processRewardsAndPenalties	201.97 ms/op	180.73 ms/op	1.12
pyrmont_e62330 - altair processRegistryUpdates	11.987 us/op	20.188 us/op	0.59
pyrmont_e62330 - altair processSlashings	854.00 ns/op	1.0480 us/op	0.81
pyrmont_e62330 - altair processEth1DataReset	893.00 ns/op	942.00 ns/op	0.95
pyrmont_e62330 - altair processEffectiveBalanceUpdates	18.961 ms/op	20.482 ms/op	0.93
pyrmont_e62330 - altair processSlashingsReset	15.719 us/op	47.755 us/op	0.33
pyrmont_e62330 - altair processRandaoMixesReset	23.248 us/op	30.231 us/op	0.77
pyrmont_e62330 - altair processHistoricalRootsUpdate	1.0290 us/op	1.2540 us/op	0.82
pyrmont_e62330 - altair processParticipationFlagUpdates	50.987 ms/op	119.26 ms/op	0.43
pyrmont_e62330 - altair processSyncCommitteeUpdates	773.00 ns/op	925.00 ns/op	0.84
pyrmont_e62330 - altair afterProcessEpoch	242.67 ms/op	230.43 ms/op	1.05
altair processInactivityUpdates - 250000 normalcase	307.54 ms/op	304.34 ms/op	1.01
altair processInactivityUpdates - 250000 worstcase	305.76 ms/op	337.59 ms/op	0.91
altair processParticipationFlagUpdates - 250000 anycase	97.169 ms/op	132.45 ms/op	0.73
altair processRewardsAndPenalties - 250000 normalcase	391.84 ms/op	379.97 ms/op	1.03
altair processRewardsAndPenalties - 250000 worstcase	352.27 ms/op	367.03 ms/op	0.96
altair processSyncCommitteeUpdates - 250000	494.13 ms/op	541.20 ms/op	0.91
Tree 40 250000 create	599.33 ms/op	784.25 ms/op	0.76
Tree 40 250000 get(125000)	1.7240 us/op	2.0104 us/op	0.86
Tree 40 250000 set(125000)	2.0189 us/op	2.6904 us/op	0.75
Tree 40 250000 toArray()	47.760 ms/op	51.879 ms/op	0.92
Tree 40 250000 iterate all - toArray() + loop	47.504 ms/op	64.936 ms/op	0.73
Tree 40 250000 iterate all - get(i)	471.21 ms/op	524.87 ms/op	0.90
MutableVector 250000 create	24.976 ms/op	26.778 ms/op	0.93
MutableVector 250000 get(125000)	15.179 ns/op	16.425 ns/op	0.92
MutableVector 250000 set(125000)	4.0557 us/op	5.4807 us/op	0.74
MutableVector 250000 toArray()	9.6132 ms/op	10.319 ms/op	0.93
MutableVector 250000 iterate all - toArray() + loop	9.6459 ms/op	9.9334 ms/op	0.97
MutableVector 250000 iterate all - get(i)	4.1262 ms/op	3.9669 ms/op	1.04
Array 250000 create	6.6531 ms/op	7.3150 ms/op	0.91
Array 250000 clone - spread	3.5374 ms/op	2.7706 ms/op	1.28
Array 250000 get(125000)	0.90200 ns/op	0.98600 ns/op	0.91
Array 250000 set(125000)	0.91100 ns/op	1.0040 ns/op	0.91
Array 250000 iterate all - loop	140.48 us/op	154.48 us/op	0.91
aggregationBits - 2048 els - readonlyValues	377.44 us/op	345.29 us/op	1.09
aggregationBits - 2048 els - zipIndexesInBitList	42.525 us/op	43.318 us/op	0.98
ssz.Root.equals	1.9790 us/op	2.2760 us/op	0.87
ssz.Root.equals with valueOf()	2.4410 us/op	2.3910 us/op	1.02
byteArrayEquals with valueOf()	2.5470 us/op	2.4620 us/op	1.03
phase0 processBlock - 250000 vs - 7PWei normalcase	23.472 ms/op	21.876 ms/op	1.07
phase0 processBlock - 250000 vs - 7PWei worstcase	124.43 ms/op	132.55 ms/op	0.94
phase0 afterProcessEpoch - 250000 vs - 7PWei	235.69 ms/op	262.49 ms/op	0.90
phase0 beforeProcessEpoch - 250000 vs - 7PWei	927.23 ms/op	1.0071 s/op	0.92
phase0 processEpoch - mainnet_e58758	1.2957 s/op	1.4411 s/op	0.90
mainnet_e58758 - phase0 beforeProcessEpoch	765.83 ms/op	788.49 ms/op	0.97
mainnet_e58758 - phase0 processJustificationAndFinalization	151.88 us/op	163.72 us/op	0.93
mainnet_e58758 - phase0 processRewardsAndPenalties	291.19 ms/op	317.43 ms/op	0.92
mainnet_e58758 - phase0 processRegistryUpdates	119.22 us/op	140.60 us/op	0.85
mainnet_e58758 - phase0 processSlashings	856.00 ns/op	1.4150 us/op	0.60
mainnet_e58758 - phase0 processEth1DataReset	833.00 ns/op	1.1960 us/op	0.70
mainnet_e58758 - phase0 processEffectiveBalanceUpdates	29.904 ms/op	82.908 ms/op	0.36
mainnet_e58758 - phase0 processSlashingsReset	15.948 us/op	46.757 us/op	0.34
mainnet_e58758 - phase0 processRandaoMixesReset	26.657 us/op	58.445 us/op	0.46
mainnet_e58758 - phase0 processHistoricalRootsUpdate	1.2710 us/op	1.4470 us/op	0.88
mainnet_e58758 - phase0 processParticipationRecordUpdates	15.852 us/op	24.792 us/op	0.64
mainnet_e58758 - phase0 afterProcessEpoch	207.21 ms/op	234.16 ms/op	0.88
phase0 processEffectiveBalanceUpdates - 250000 normalcase	43.039 ms/op	48.915 ms/op	0.88
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5	7.6812 s/op	8.8292 s/op	0.87
phase0 processRegistryUpdates - 250000 normalcase	305.64 us/op	874.38 us/op	0.35
phase0 processRegistryUpdates - 250000 badcase_full_deposits	11.499 ms/op	17.446 ms/op	0.66
phase0 processRegistryUpdates - 250000 worstcase 0.5	6.5691 s/op	7.5210 s/op	0.87
phase0 getAttestationDeltas - 250000 normalcase	57.345 ms/op	79.037 ms/op	0.73
phase0 getAttestationDeltas - 250000 worstcase	71.091 ms/op	56.564 ms/op	1.26
phase0 processSlashings - 250000 worstcase	8.7430 us/op	11.191 us/op	0.78
shuffle list - 16384 els	14.366 ms/op	15.112 ms/op	0.95
shuffle list - 250000 els	200.47 ms/op	219.50 ms/op	0.91
getPubkeys - index2pubkey - req 1000 vs - 250000 vc	2.6751 ms/op	2.8517 ms/op	0.94
getPubkeys - validatorsArr - req 1000 vs - 250000 vc	505.08 us/op	525.08 us/op	0.96
getPubkeys - persistent - req 1000 vs - 250000 vc	18.496 us/op	19.594 us/op	0.94
BLS verify - blst-native	2.7122 ms/op	2.3773 ms/op	1.14
BLS verifyMultipleSignatures 3 - blst-native	5.4894 ms/op	5.0031 ms/op	1.10
BLS verifyMultipleSignatures 8 - blst-native	11.785 ms/op	10.736 ms/op	1.10
BLS verifyMultipleSignatures 32 - blst-native	43.075 ms/op	37.801 ms/op	1.14
BLS aggregatePubkeys 32 - blst-native	60.254 us/op	51.968 us/op	1.16
BLS aggregatePubkeys 128 - blst-native	224.78 us/op	210.87 us/op	1.07
getAttestationsForBlock	97.605 ms/op	112.47 ms/op	0.87
CheckpointStateCache - add get delete	26.852 us/op	30.713 us/op	0.87
validate gossip signedAggregateAndProof - struct	6.3322 ms/op	5.8797 ms/op	1.08
validate gossip signedAggregateAndProof - treeBacked	6.6830 ms/op	5.9372 ms/op	1.13
validate gossip attestation - struct	3.0183 ms/op	2.8224 ms/op	1.07
validate gossip attestation - treeBacked	3.0569 ms/op	2.8453 ms/op	1.07

by benchmarkbot/action