Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

v0.39.0 #4217

Merged
merged 42 commits into from
Jun 30, 2022
Merged

v0.39.0 #4217

merged 42 commits into from
Jun 30, 2022

Conversation

wemeetagain
Copy link
Member

No description provided.

dapplion and others added 30 commits June 3, 2022 13:18
@dapplion
Populate eth1 deposit cache with remote follow distance timed based (#…
2f0c2d7 
…4111)
@dapplion
Fix HttpClient event handler leak (#4107)
a61e8a5 
Fix leak in signalGlobal event listeners
@dapplion
Add Eth1DepositTracker metrics (#4110)
c8d5c56 
* Add Eth1DepositTracker metrics

* Add Eth1DepositTracker metrics

* Remove duplicate line
@twoeths @dapplion
Use async version of crypto.randomBytes() (#4075)
adc9906 
* Use async version of crypto.randomBytes()

* Use promisify node util

* Compute randomBytesAsync once

Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
@g11tech
Use justified blocks execution payload hash as safeBlockHash in engin…
b5e24f7 
…e fcU (#4080)

* Use justified blocks execution payload hash as safeBlockHash in engine fcU

* fix test

* safe block hash fix

* run merge ci on latest
@philknows @wemeetagain
Remove NPM instructions and add disclaimer (#4115)
ebf557d 
* Remove NPM instructions and add disclaimer

* Update docs/installation.md

Co-authored-by: Cayman <caymannava@gmail.com>
@dapplion
Remove NPM install instructions from README (#4119)
fe9d85d
@dapplion
Deprecate accounts CLI (#4104)
f74f9ef 
* Deprecate account cli cmd

* Merge options

* Delete stable account e2e test

* Log error.message not the error object

* Review PR
@dapplion
Flat state transition across forks (#4120)
fe5c98e 
* Update src

* Update tests

* Update spec tests

* Fix Lodestar src

* Fix type issues

* Fix perf test types

* Review PR

* Remove un-used import

* Don't check state root in computeNewStateRoot

* Rename slashValidator
@wemeetagain @g11tech @dapplion
Stable (#4129)
56a6bb4 
* v0.38.0

* Populate eth1 deposit cache with remote follow distance timed based (#4111)

* Fix HttpClient event handler leak (#4107)

Fix leak in signalGlobal event listeners

* Add Eth1DepositTracker metrics (#4110)

* Add Eth1DepositTracker metrics

* Add Eth1DepositTracker metrics

* Remove duplicate line

* Fix publish stable workflow (#4130)

Co-authored-by: harkamal <gajinder@g11.in>
Co-authored-by: Lion - dapplion <35266934+dapplion@users.noreply.github.com>
@dapplion
Add JSON RPC client metrics (#4127)
d716b07 
* Add JsonRpcHttpClient metrics

* JsonRpcHttpClient metrics add routeId

* Update restApiClient metrics definition
@dapplion
Add opPool metrics (#4126)
a1a740b 
* Add opPool metrics

* Use . operator
@dapplion
Add bls.aggregatedPubkeys metric (#4125)
ff0abc6
@dapplion
Add reorg and parent distance metrics (#4123)
2deb6a2 
* Add forkChoiceReorgDistance metric

* Add parentBlockDistance metric
@wemeetagain @dapplion
Add reqResp request time metrics (#4131)
8015aba 
Co-authored-by: dapplion <35266934+dapplion@users.noreply.github.com>
@g11tech
Update mainnet shadow fork 6 and ropsten configs and images (#4132)
719d936 
* mainnet shadow fork 6 config with small robustness update for cl

* config and image update

* update readme with the standard geth image

* remove the custom dockerfile
@twoeths
Skip checking known participants when publishing ContributionAndProof (…
a3e286f 
…#4097)

* Skip checking known participants when publishing ContributionAndProof

* chore: merge if statements
@dapplion
Add new CLI tool: flare (#4101)
4bbe4a3 
* Add bcli package

* Use consistent format for dev index range

* Dedup code with deriveSecretKeys

* Fix argument parsing

* cap batchSize by MAX_VALIDATORS_PER_COMMITTEE

* Fix check-build script

* Add unit test

* Rename to flare
@twoeths
Improve UnknownBlock sync (#4134)
f16a76d 
* UnknownBlock sync: only download non-finalized blocks

* chore: add parentRoot to log
@dapplion
Apply preset to ENV from CLI args (#4136)
8535830 
* Apply preset to ENV from CLI args

* Add rationale
@dapplion
Handle multiple own sync committee aggregator per subnet (#4135)
c9eea32 
* Explain selectionProofs array

* Remove lowestPermissibleSlot variable

* Handle multiple validators in sync committee subnet

* Fix tests
@dapplion
Improve DB metrics (#4121)
1f5fe90 
* Improve db metrics

* Wire new db metrics to lodestar

* Connect metrics to validator

* Remove unused variable
@dapplion
Support gnosis beacon chain with --network gnosis (#4137)
6dd408b 
* Support gnosis beacon chain with --network gnosis

* Fix tests

* Fix export
@dapplion
Remove unnecessary dependency from flare (#4139)
4c439ad
@dapplion
Add space to POS banner (#4141)
43efd05
@dapplion
Declare routeId in HTTP client metrics (#4151)
32e7ce3 
Declare routeId in http client metrics
@claravanstaden
Export lodestar package modules (#4150)
ceba4a9 
* Exports modules required for script.

* Exports lodestar modules.

* Adds the rest of the lodestar package modules.

Co-authored-by: claravanstaden <Cats 4 life!>
@philknows
Docs structure update and addition of post-merge feeRecipient and JWT…
ca94066 
… instructions (#4118)

* Init update key-management.md

* Reconfigure docs and add jwt fee recipient

* Add keystore folder info to validator-management

* Add link to comprehensive setup guide for docker

* Improved NPM warning

* Add flare to libraries list on docs

* Corrections to beacon-management docs

* Align readme and docs index
@dapplion
Remove wtfnode (#4156)
04155c6 
* Remove wtfnode

* Remove getWtfNode route
@dapplion
Split lodestar-api by APIs (#4158)
9feb5bd 
* Modularize lodestar-api

* Fix imports

* Move beacon API interface to routes

* Move tests to same src dir structure
g11tech and others added 11 commits June 15, 2022 13:53
@g11tech
Avoid json parsing an expected empty response (#4163)
e5b74d5 
* Avoid json parsing an expected empty response

* small comment
@dapplion
Add Sepolia config (#4164)
851d698
@dadepo
move the import/no-extraneous-dependencies back to correct placement
a4c0263
@wemeetagain
Merge pull request #4161 from ChainSafe/dadepo/return-fastify-to-dev-dep
83aa2cc 
Have fastify back as dev dependency in lodestar-api
@g11tech
Hotfix concurrent validators decrypt OOM issue (#4174)
cbd8a1c
@dapplion
Merge remote-tracking branch 'origin/stable' into unstable
2eeb965
@dadepo
Turn off the import/no-named-as-default-member in tests (#4162)
9b3ac42 
* turn off the import/no-named-as-default-member in tests

* Added comments
@wemeetagain
v0.39.0
9b76bdc
@g11tech @wemeetagain
Add ability to spin up sepolia network via merge setup (#4193)
675a3ad 
* Add ability to spin up sepolia network via merge setup

* sepolia network arg for besu
@g11tech @wemeetagain
Relocate/promote merge devnet scripts as merge scripts (#4202)
1298ceb 
* Relocate/promote merge devnet scripts as merge scripts

* some more readme clarifications
@g11tech @wemeetagain
Update sepolia ttd (#4215)
55e7ca2
@github-actions
Copy link
Contributor

github-actions bot commented Jun 28, 2022 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 1979216 Previous: - Ratio
altair processAttestation - 250000 vs - 7PWei normalcase 3.3702 ms/op
altair processAttestation - 250000 vs - 7PWei worstcase 5.2110 ms/op
altair processAttestation - setStatus - 1/6 committees join 168.13 us/op
altair processAttestation - setStatus - 1/3 committees join 336.40 us/op
altair processAttestation - setStatus - 1/2 committees join 489.37 us/op
altair processAttestation - setStatus - 2/3 committees join 646.12 us/op
altair processAttestation - setStatus - 4/5 committees join 904.64 us/op
altair processAttestation - setStatus - 100% committees join 1.0923 ms/op
altair processBlock - 250000 vs - 7PWei normalcase 22.117 ms/op
altair processBlock - 250000 vs - 7PWei normalcase hashState 32.179 ms/op
altair processBlock - 250000 vs - 7PWei worstcase 77.916 ms/op
altair processBlock - 250000 vs - 7PWei worstcase hashState 95.262 ms/op
phase0 processBlock - 250000 vs - 7PWei normalcase 3.2815 ms/op
phase0 processBlock - 250000 vs - 7PWei worstcase 51.044 ms/op
altair processEth1Data - 250000 vs - 7PWei normalcase 691.43 us/op
Tree 40 250000 create 755.55 ms/op
Tree 40 250000 get(125000) 250.71 ns/op
Tree 40 250000 set(125000) 2.1999 us/op
Tree 40 250000 toArray() 27.209 ms/op
Tree 40 250000 iterate all - toArray() + loop 27.410 ms/op
Tree 40 250000 iterate all - get(i) 111.54 ms/op
MutableVector 250000 create 13.475 ms/op
MutableVector 250000 get(125000) 10.850 ns/op
MutableVector 250000 set(125000) 540.38 ns/op
MutableVector 250000 toArray() 6.1132 ms/op
MutableVector 250000 iterate all - toArray() + loop 6.2254 ms/op
MutableVector 250000 iterate all - get(i) 2.8413 ms/op
Array 250000 create 5.9516 ms/op
Array 250000 clone - spread 3.2576 ms/op
Array 250000 get(125000) 1.6130 ns/op
Array 250000 set(125000) 1.5600 ns/op
Array 250000 iterate all - loop 151.25 us/op
effectiveBalanceIncrements clone Uint8Array 300000 53.807 us/op
effectiveBalanceIncrements clone MutableVector 300000 681.00 ns/op
effectiveBalanceIncrements rw all Uint8Array 300000 247.22 us/op
effectiveBalanceIncrements rw all MutableVector 300000 134.96 ms/op
phase0 afterProcessEpoch - 250000 vs - 7PWei 185.93 ms/op
phase0 beforeProcessEpoch - 250000 vs - 7PWei 58.203 ms/op
altair processEpoch - mainnet_e81889 533.32 ms/op
mainnet_e81889 - altair beforeProcessEpoch 121.39 ms/op
mainnet_e81889 - altair processJustificationAndFinalization 17.234 us/op
mainnet_e81889 - altair processInactivityUpdates 9.9405 ms/op
mainnet_e81889 - altair processRewardsAndPenalties 108.58 ms/op
mainnet_e81889 - altair processRegistryUpdates 2.6480 us/op
mainnet_e81889 - altair processSlashings 628.00 ns/op
mainnet_e81889 - altair processEth1DataReset 626.00 ns/op
mainnet_e81889 - altair processEffectiveBalanceUpdates 2.1734 ms/op
mainnet_e81889 - altair processSlashingsReset 4.1900 us/op
mainnet_e81889 - altair processRandaoMixesReset 4.2670 us/op
mainnet_e81889 - altair processHistoricalRootsUpdate 708.00 ns/op
mainnet_e81889 - altair processParticipationFlagUpdates 3.5700 us/op
mainnet_e81889 - altair processSyncCommitteeUpdates 715.00 ns/op
mainnet_e81889 - altair afterProcessEpoch 216.63 ms/op
phase0 processEpoch - mainnet_e58758 515.67 ms/op
mainnet_e58758 - phase0 beforeProcessEpoch 179.09 ms/op
mainnet_e58758 - phase0 processJustificationAndFinalization 15.017 us/op
mainnet_e58758 - phase0 processRewardsAndPenalties 92.799 ms/op
mainnet_e58758 - phase0 processRegistryUpdates 7.4280 us/op
mainnet_e58758 - phase0 processSlashings 620.00 ns/op
mainnet_e58758 - phase0 processEth1DataReset 584.00 ns/op
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.6288 ms/op
mainnet_e58758 - phase0 processSlashingsReset 4.3060 us/op
mainnet_e58758 - phase0 processRandaoMixesReset 4.1780 us/op
mainnet_e58758 - phase0 processHistoricalRootsUpdate 670.00 ns/op
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.3910 us/op
mainnet_e58758 - phase0 afterProcessEpoch 162.97 ms/op
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.9386 ms/op
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.1467 ms/op
altair processInactivityUpdates - 250000 normalcase 21.770 ms/op
altair processInactivityUpdates - 250000 worstcase 31.103 ms/op
phase0 processRegistryUpdates - 250000 normalcase 5.9150 us/op
phase0 processRegistryUpdates - 250000 badcase_full_deposits 362.32 us/op
phase0 processRegistryUpdates - 250000 worstcase 0.5 173.30 ms/op
altair processRewardsAndPenalties - 250000 normalcase 93.973 ms/op
altair processRewardsAndPenalties - 250000 worstcase 89.937 ms/op
phase0 getAttestationDeltas - 250000 normalcase 11.847 ms/op
phase0 getAttestationDeltas - 250000 worstcase 11.900 ms/op
phase0 processSlashings - 250000 worstcase 4.9441 ms/op
altair processSyncCommitteeUpdates - 250000 287.60 ms/op
BeaconState.hashTreeRoot - No change 625.00 ns/op
BeaconState.hashTreeRoot - 1 full validator 66.153 us/op
BeaconState.hashTreeRoot - 32 full validator 652.35 us/op
BeaconState.hashTreeRoot - 512 full validator 7.2207 ms/op
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 89.500 us/op
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.2370 ms/op
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 16.638 ms/op
BeaconState.hashTreeRoot - 1 balances 70.710 us/op
BeaconState.hashTreeRoot - 32 balances 620.67 us/op
BeaconState.hashTreeRoot - 512 balances 6.4925 ms/op
BeaconState.hashTreeRoot - 250000 balances 97.126 ms/op
aggregationBits - 2048 els - zipIndexesInBitList 27.657 us/op
regular array get 100000 times 63.225 us/op
wrappedArray get 100000 times 63.677 us/op
arrayWithProxy get 100000 times 28.981 ms/op
ssz.Root.equals 543.00 ns/op
byteArrayEquals 541.00 ns/op
shuffle list - 16384 els 11.377 ms/op
shuffle list - 250000 els 166.12 ms/op
processSlot - 1 slots 13.308 us/op
processSlot - 32 slots 1.9904 ms/op
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 370.19 us/op
getCommitteeAssignments - req 1 vs - 250000 vc 5.4655 ms/op
getCommitteeAssignments - req 100 vs - 250000 vc 7.9862 ms/op
getCommitteeAssignments - req 1000 vs - 250000 vc 8.5960 ms/op
computeProposers - vc 250000 18.803 ms/op
computeEpochShuffling - vc 250000 170.40 ms/op
getNextSyncCommittee - vc 250000 287.93 ms/op
pass gossip attestations to forkchoice per slot 5.1570 ms/op
computeDeltas 3.2052 ms/op
computeProposerBoostScoreFromBalances 806.93 us/op
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.0413 ms/op
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 71.326 us/op
BLS verify - blst-native 2.1670 ms/op
BLS verifyMultipleSignatures 3 - blst-native 4.4718 ms/op
BLS verifyMultipleSignatures 8 - blst-native 9.6858 ms/op
BLS verifyMultipleSignatures 32 - blst-native 35.236 ms/op
BLS aggregatePubkeys 32 - blst-native 46.687 us/op
BLS aggregatePubkeys 128 - blst-native 182.19 us/op
getAttestationsForBlock 52.159 ms/op
isKnown best case - 1 super set check 487.00 ns/op
isKnown normal case - 2 super set checks 478.00 ns/op
isKnown worse case - 16 super set checks 478.00 ns/op
CheckpointStateCache - add get delete 10.988 us/op
validate gossip signedAggregateAndProof - struct 5.0087 ms/op
validate gossip attestation - struct 2.3659 ms/op
altair verifyImport mainnet_s3766816:31 6.8211 s/op
pickEth1Vote - no votes 2.2386 ms/op
pickEth1Vote - max votes 21.834 ms/op
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.594 ms/op
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 21.845 ms/op
pickEth1Vote - Eth1Data fastSerialize value x2048 1.5246 ms/op
pickEth1Vote - Eth1Data fastSerialize tree x2048 14.947 ms/op
bytes32 toHexString 1.1600 us/op
bytes32 Buffer.toString(hex) 853.00 ns/op
bytes32 Buffer.toString(hex) from Uint8Array 1.1080 us/op
bytes32 Buffer.toString(hex) + 0x 880.00 ns/op
Object access 1 prop 0.46900 ns/op
Map access 1 prop 0.38200 ns/op
Object get x1000 11.002 ns/op
Map get x1000 1.1360 ns/op
Object set x1000 78.927 ns/op
Map set x1000 51.785 ns/op
Return object 10000 times 0.45160 ns/op
Throw Error 10000 times 5.9384 us/op
enrSubnets - fastDeserialize 64 bits 3.0170 us/op
enrSubnets - ssz BitVector 64 bits 977.00 ns/op
enrSubnets - fastDeserialize 4 bits 489.00 ns/op
enrSubnets - ssz BitVector 4 bits 994.00 ns/op
prioritizePeers score -10:0 att 32-0.1 sync 2-0 87.686 us/op
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 119.18 us/op
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 210.63 us/op
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 412.17 us/op
prioritizePeers score 0:0 att 64-1 sync 4-1 419.85 us/op
RateTracker 1000000 limit, 1 obj count per request 187.10 ns/op
RateTracker 1000000 limit, 2 obj count per request 138.78 ns/op
RateTracker 1000000 limit, 4 obj count per request 114.58 ns/op
RateTracker 1000000 limit, 8 obj count per request 101.83 ns/op
RateTracker with prune 4.5130 us/op
array of 16000 items push then shift 51.614 us/op
LinkedList of 16000 items push then shift 17.444 ns/op
array of 16000 items push then pop 206.81 ns/op
LinkedList of 16000 items push then pop 14.568 ns/op
array of 24000 items push then shift 77.381 us/op
LinkedList of 24000 items push then shift 22.400 ns/op
array of 24000 items push then pop 203.03 ns/op
LinkedList of 24000 items push then pop 19.192 ns/op
intersect bitArray bitLen 8 10.911 ns/op
intersect array and set length 8 169.74 ns/op
intersect bitArray bitLen 128 55.915 ns/op
intersect array and set length 128 1.8757 us/op

by benchmarkbot/action

@wemeetagain wemeetagain marked this pull request as ready for review June 30, 2022 18:52
@wemeetagain wemeetagain requested a review from a team as a code owner June 30, 2022 18:52
@wemeetagain
Copy link
Member Author

We got agreement in discord that this release is ready to go.

dapplion
dapplion approved these changes Jun 30, 2022
View reviewed changes
Copy link
Contributor

@dapplion dapplion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metrics look good!

@wemeetagain wemeetagain merged commit 5d3d940 into stable Jun 30, 2022
@wemeetagain wemeetagain deleted the rc/v0.39.0 branch June 30, 2022 19:00
@snyk-bot snyk-bot mentioned this pull request Nov 19, 2022
Closed
@nazarhussain nazarhussain mentioned this pull request Nov 22, 2022
Closed
@snyk-bot snyk-bot mentioned this pull request Dec 5, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dapplion dapplion dapplion approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@wemeetagain @dapplion @twoeths @g11tech @philknows @claravanstaden @dadepo