Align code base to v0.11 #5127

terencechain · 2020-03-17T23:44:13Z

Builds on #4804 and incorporates all changes.

Resolves #5119

* Refactoring of initial sync (#5096) * implements blocks queue * refactors updateCounter method * fixes deadlock on stop w/o start * refactors updateSchedulerState * more tests on schduler * parseFetchResponse tests * wraps up tests for blocks queue * eod commit * fixes data race in round robin * revamps fetcher * fixes race conditions + livelocks + deadlocks * less verbose output * fixes data race, by isolating critical sections * minor refactoring: resolves blocking calls * implements init-sync queue * udpate fetch/send buffers in blocks fetcher * blockState enum-like type alias * refactors common code into releaseTicket() * better gc * linter * minor fix to round robin * moves original round robin into its own package * adds enableInitSyncQueue flag * fixes issue with init-sync service selection * Update beacon-chain/sync/initial-sync/round_robin.go Co-Authored-By: terence tsao <terence@prysmaticlabs.com> * initsyncv1 -> initsyncold * adds span Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com> Co-authored-by: terence tsao <terence@prysmaticlabs.com> * Handle rewards overflow * Revert "Refactoring of initial sync (#5096)" This reverts commit 3ec2a0f. Co-authored-by: Victor Farazdagi <simple.square@gmail.com> Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com> Co-authored-by: Raul Jordan <raul@prysmaticlabs.com>

beacon-chain/core/blocks/block_operations_fuzz_test.go

beacon-chain/core/blocks/block_operations_test.go

beacon-chain/core/blocks/eth1_data_test.go

beacon-chain/core/helpers/signing_root.go

0xKiwi · 2020-04-14T00:16:19Z

beacon-chain/db/kv/check_historical_state.go

-		deniedText := "Historical states will not be generated. Please remove usage --new-state-mgmt"
+		actionText := "--disable-new-state-mgmt was used. To proceed without the flag, the db will need " +
+			"to generate and save historical states. This process may take a while, - do you want to proceed? (Y/N)"
+		deniedText := "Historical states will not be generated. Please continue use --disable-new-state-mgmt"


What is this supposed to say @terencechain , "please continue use --disable" on the denied text?

Huh? The PR says Please continue use --disable-new-state-mgmt

The english just seems weird, its on the denied text for a confirmation of the usage of the flag. "Please continue use --disable" when you deny the flag? Maybe its supposed to be "Please continue using --disable"?

sure we can update it

0xKiwi · 2020-04-14T00:43:19Z

shared/interop/generate_genesis_state.go

@@ -149,8 +150,15 @@ func createDepositData(privKey *bls.SecretKey, pubKey *bls.PublicKey) (*ethpb.De
 	if err != nil {
 		return nil, err
 	}
-	domain := bls.ComputeDomain(params.BeaconConfig().DomainDeposit)
-	di.Signature = privKey.Sign(sr[:], domain).Marshal()
+	domain, err := helpers.ComputeDomain(params.BeaconConfig().DomainDeposit, nil, nil)


Should this be changed?

0xKiwi · 2020-04-14T00:43:53Z

shared/keystore/deposit_input_test.go

 	"github.com/prysmaticlabs/prysm/shared/keystore"
 	"github.com/prysmaticlabs/prysm/shared/params"
 )

 func TestDepositInput_GeneratesPb(t *testing.T) {
+	t.Skip("To be resolved until 5119 gets in")


Does this need to stay skipped?

No, please unskip it

shared/testutil/helpers.go

0xKiwi · 2020-04-14T00:52:52Z

proto/beacon/rpc/v1/BUILD.bazel

+
+go_library(
+    name = "go_default_library",
+    srcs = ["slasher.pb.go"],


Not sure, must've missed them in #5308 , on it.

0xKiwi · 2020-04-14T00:53:54Z

proto/beacon/rpc/v1/slasher.pb.go

@@ -0,0 +1,2042 @@
+// Code generated by protoc-gen-gogo. DO NOT EDIT.
+// source: proto/beacon/rpc/v1/slasher.proto


This is no longer needed, we can remove. On it.

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

* address all comments * set faucet * nishant feedback * Update beacon-chain/p2p/service.go Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* Revert "Updates for remote keymanager (#5260)" This reverts commit bbcd895. * Revert "Remove keystore keymanager from validator (#5236)" This reverts commit 4600877. * Revert "Update eth2 wallet keymanager (#4984)" This reverts commit 7f7ef43. Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

* remove duplicated BLS, add golang.org/x/mod * Update BLS and restrict visibility * fix build

* Unskip benchutil tests * Remove protos and gaz * Fixes

* check * fix test * fix size * fix test * more fixes * fix test again

* Proper err handling for tests * Lint * Fixed rest of the tests * Gaz * Fixed old master tests

prestonvanloon

LGTM

protolambda

A copy paste error caused two critical bugs, and tests didn't catch it sadly. Refactoring rewards in spec to be less tangled together, then tested with good encapsulation, would be good. Too many reward bugs so far :(

Also, two more possible bugs:

Attester eligibility should be considered in rewards and penalties processing. Currently the whole registry is used.
Order of operations wrong, the rewards overflow spec bug is still there in Prysm. And the test that was supposed to catch it didn't

And maxAtteserReward typo.

protolambda · 2020-04-23T10:42:44Z

beacon-chain/core/epoch/precompute/reward_penalty.go

@@ -93,14 +95,18 @@ func attestationDelta(state *stateTrie.BeaconState, bp *Balance, v *Validator) (

 	// Process target reward / penalty
 	if v.IsPrevEpochTargetAttester && !v.IsSlashed {
-		r += br * bp.PrevEpochTargetAttesters / bp.CurrentEpoch
+		inc := params.BeaconConfig().EffectiveBalanceIncrement
+		rewardNumerator := br * bp.PrevEpochAttesters / inc


This is a critical bug 🪲 🐛

protolambda · 2020-04-23T10:42:56Z

beacon-chain/core/epoch/precompute/reward_penalty.go

 	} else {
 		p += br
 	}

 	// Process head reward / penalty
 	if v.IsPrevEpochHeadAttester && !v.IsSlashed {
-		r += br * bp.PrevEpochHeadAttesters / bp.CurrentEpoch
+		inc := params.BeaconConfig().EffectiveBalanceIncrement
+		rewardNumerator := br * bp.PrevEpochAttesters / inc


This is a critical bug as well 🪲 🐛

prylabs-bulldozer bot and others added 30 commits March 14, 2020 15:34

Merge refs/heads/master into v0.10.1

f1f536a

Merge refs/heads/master into v0.10.1

abfbd15

Merge refs/heads/master into v0.10.1

b1e5bae

Merge refs/heads/master into v0.10.1

2052739

Merge refs/heads/master into v0.10.1

5a1b6f1

Merge refs/heads/master into v0.10.1

20cbfe6

Merge refs/heads/master into v0.10.1

78c0dbc

Merge refs/heads/master into v0.10.1

1948138

Merge refs/heads/master into v0.10.1

830ff47

Merge refs/heads/master into v0.10.1

7c44117

Merge refs/heads/master into v0.10.1

54df077

Merge refs/heads/master into v0.10.1

4b6a300

Merge refs/heads/master into v0.10.1

e24b356

Merge refs/heads/master into v0.10.1

863fef0

Merge refs/heads/master into v0.10.1

bd935ef

Merge refs/heads/master into v0.10.1

0991a6b

add in new patch and workspace

de2e08f

update cloners

c6f1b8f

updated block operations

7057c0e

updated validator client

3ac67c2

Merge refs/heads/master into v0.10.1

30a171e

updated block operations test

73d7e04

skip benchmark test

09ad93e

updated transition test

dadbf9a

updated db kv tests

67f2a1c

updated ops tests

ec470a3

updated ops tests

15ffdee

updated slashing tests

81eadbd

updated rpc tests

ac27882

0xKiwi reviewed Apr 14, 2020

View reviewed changes

terencechain and others added 11 commits April 13, 2020 18:07

Update beacon-chain/core/blocks/block_operations_fuzz_test.go

1e19c7f

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

Update beacon-chain/core/blocks/block_operations_test.go

f695932

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

Update shared/testutil/helpers.go

b435a4a

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

Update beacon-chain/core/helpers/signing_root.go

de7195c

Co-Authored-By: Ivan Martinez <ivanthegreatdev@gmail.com>

Resolve Misc v0.11 Items (Raul) (#5414)

0a64620

* address all comments * set faucet * nishant feedback * Update beacon-chain/p2p/service.go Co-authored-by: prylabs-bulldozer[bot] <58059840+prylabs-bulldozer[bot]@users.noreply.github.com>

Update BLS and limit visibility (#5415)

3f882af

* remove duplicated BLS, add golang.org/x/mod * Update BLS and restrict visibility * fix build

Fix eth1data test and fix order of ops (#5413)

95e44ba

use multiaddr builder (#5419)

a4f47b9

Unskip benchutil and minor v0.11 fixes (#5417)

da28c1f

* Unskip benchutil tests * Remove protos and gaz * Fixes

Networking Fixes (#5421)

2e1ebc9

* check * fix test * fix size * fix test * more fixes * fix test again

prestonvanloon mentioned this pull request Apr 14, 2020

Reintroduce keymanager updates / improvements / changes from v0.11 #5423

Closed

4 tasks

prestonvanloon added Ready For Review and removed In Progress labels Apr 14, 2020

prestonvanloon previously approved these changes Apr 14, 2020

View reviewed changes

Update ethereum APIs with latest master

f7a2aa5

prestonvanloon dismissed their stale review via f7a2aa5 April 14, 2020 19:06

terencechain added 3 commits April 14, 2020 13:05

Error handling for v0.11 tests (#5428)

6722667

* Proper err handling for tests * Lint * Fixed rest of the tests * Gaz * Fixed old master tests

Sync with master

3500f91

Rm old aggregate_test.go

7d132ea

prestonvanloon approved these changes Apr 14, 2020

View reviewed changes

prestonvanloon added the OK to merge label Apr 14, 2020

prylabs-bulldozer bot merged commit cb045dd into master Apr 14, 2020

delete-merged-branch bot deleted the v0.11 branch April 14, 2020 20:27

farazdagi mentioned this pull request Apr 16, 2020

nil attestation tests added #5457

Merged

protolambda reviewed Apr 23, 2020

View reviewed changes

protolambda mentioned this pull request Apr 23, 2020

Rewards penalty bugs and resolution steps #5593

Closed

nisdas mentioned this pull request Oct 5, 2020

[Mega tracking] - Audit Resolutions #6327

Closed

61 tasks

terencechain mentioned this pull request Dec 31, 2020

Attest early if head updates before 4 seconds #8185

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Align code base to v0.11 #5127

Align code base to v0.11 #5127

terencechain commented Mar 17, 2020 •

edited

Loading

0xKiwi Apr 14, 2020

terencechain Apr 14, 2020

0xKiwi Apr 14, 2020

terencechain Apr 14, 2020

0xKiwi Apr 14, 2020

nisdas Apr 14, 2020

0xKiwi Apr 14, 2020

terencechain Apr 14, 2020

0xKiwi Apr 14, 2020

0xKiwi Apr 14, 2020

0xKiwi Apr 14, 2020

prestonvanloon left a comment

protolambda left a comment

protolambda Apr 23, 2020

protolambda Apr 23, 2020

		@@ -0,0 +1,2042 @@
		// Code generated by protoc-gen-gogo. DO NOT EDIT.
		// source: proto/beacon/rpc/v1/slasher.proto

Align code base to v0.11 #5127

Align code base to v0.11 #5127

Conversation

terencechain commented Mar 17, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

prestonvanloon left a comment

Choose a reason for hiding this comment

protolambda left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

terencechain commented Mar 17, 2020 •

edited

Loading