Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Consensus] Refactor for guarantee signer indices (4/4) #2204

Merged

Conversation

zhangchiqing
Copy link
Member

Some refactors to guarantee signer indices PR from Alex.

@@ -5,7 +5,18 @@ import (
"fmt"
)

var ErrInsufficientShares = errors.New("insufficient threshold signature shares")
var (
ErrInvalidSignatureFormat = errors.New("signature's binary format is invalid")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added more sentinal errors

@zhangchiqing zhangchiqing changed the title Refactor for guarantee signer indices [Consensus] Refactor for guarantee signer indices Mar 25, 2022
// part of the node's internal work to generate messages. Hence, the inputs to this method come from other
// trusted components within the node. Therefore, any illegal input is treated as a symptom of an internal bug.
func EncodeSignerToIndicesAndSigType(
canonicalIdentifiers flow.IdentifierList,
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed #2101 (comment)

if len(signers) == 0 {
return fmt.Errorf("empty list of signers: %w", model.ErrInvalidFormat)
return model.NewInsufficientSignaturesErrorf("empty list of signers")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressing #2101 (comment)

Comment on lines +70 to +76
if len(stakingSignersLookup) != len(stakingSigners) {
return nil, nil, fmt.Errorf("duplicated entries in staking signers %v", stakingSignersLookup)
}
beaconSignersLookup := beaconSigners.Lookup()
if len(beaconSignersLookup) != len(beaconSigners) {
return nil, nil, fmt.Errorf("duplicated entries in beacon signers %v", stakingSignersLookup)
}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address #2101 (comment)

signerIndices = bitutils.MakeBitVector(len(canonicalIdentifiers))
for canonicalIdx, member := range canonicalIdentifiers {
if _, ok := signersLookup[member]; ok {
bitutils.SetBit(signerIndices, canonicalIdx)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressing #2101 (comment)

// Expected Error returns during normal operations:
// * IncompatibleIndexSliceError indicates that `signerIndices` has the wrong length
// * IllegallyPaddedIndexSliceError is the vector is padded with bits other than 0
func DecodeSignerIndicesToIdentities(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressing #2140 (comment)

@zhangchiqing zhangchiqing changed the title [Consensus] Refactor for guarantee signer indices [Consensus] Refactor for guarantee signer indices (4/4) Mar 25, 2022
@zhangchiqing zhangchiqing requested review from tarakby, jordanschalm and durkmurder and removed request for m4ksio, vishalchangrani and ramtinms March 25, 2022 16:32
signerCounter := 0
for canonicalIdx, member := range canonicalIdentifiers {
if _, ok := stakingSignersLookup[member]; ok {
bitutils.SetBit(signerIndices, canonicalIdx)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zhangchiqing zhangchiqing force-pushed the leo/guarantee-signer-indices branch from 7ef0931 to 8dd8c35 Compare March 25, 2022 16:41
@zhangchiqing zhangchiqing force-pushed the leo/alex-guarantee-signer-indices branch from 3b80e9e to b28671b Compare March 25, 2022 16:43
@@ -9,8 +9,8 @@ import (

var (
ErrUnverifiableBlock = errors.New("block proposal can't be verified, because its view is above the finalized view, but its QC is below the finalized view")
ErrInvalidFormat = errors.New("invalid signature format")
ErrInvalidSignature = errors.New("invalid signature")
//ErrInvalidFormat = errors.New("invalid signature format")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//ErrInvalidFormat = errors.New("invalid signature format")

@@ -4,8 +4,9 @@ import (
"errors"
"fmt"

"github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we move it lower? to have it grouped togeter?

@@ -114,7 +116,7 @@ func (s *CombinedVoteProcessorV2TestSuite) TestProcess_InvalidSignatureFormat()

// valid length is SigLen or 2*SigLen
generator := rapid.IntRange(0, 128).Filter(func(value int) bool {
return value != hsig.SigLen && value != 2*hsig.SigLen
return value != signature2.SigLen && value != 2*signature2.SigLen
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's avoid names like signature2

@@ -7,6 +7,8 @@ import (
"testing"
"time"

signature2 "github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we already have msig

@@ -7,6 +7,9 @@ import (
"testing"
"time"

"github.com/onflow/flow-go/module/signature"
msig "github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

duplicated + can we move it lower?

@@ -4,6 +4,8 @@ import (
"errors"
"fmt"

signature2 "github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
signature2 "github.com/onflow/flow-go/module/signature"
msig "github.com/onflow/flow-go/module/signature"

@@ -3,6 +3,8 @@ package votecollector
import (
"errors"

"github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, grouping

@@ -4,10 +4,11 @@ import (
"errors"
"fmt"

"github.com/onflow/flow-go/module/signature"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

grouping

@zhangchiqing zhangchiqing force-pushed the leo/alex-guarantee-signer-indices branch from f4d1343 to 4d03608 Compare March 29, 2022 16:21
@zhangchiqing zhangchiqing force-pushed the leo/alex-guarantee-signer-indices branch from 3846b1d to d5bc29f Compare March 29, 2022 22:54
@zhangchiqing zhangchiqing merged commit f7a9f89 into leo/guarantee-signer-indices Mar 30, 2022
@zhangchiqing zhangchiqing deleted the leo/alex-guarantee-signer-indices branch March 30, 2022 20:14
zhangchiqing added a commit that referenced this pull request Mar 30, 2022
* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
zhangchiqing added a commit that referenced this pull request Mar 30, 2022
* implement IdentitiesByIndices

* convert signer index

* implement encoding and decoding

* add test cases

* remove signer indices file

* fix Packer

* add comments

* remove IdentitiesByIndices

* update mock

* fix test cases

* add comments

* remove fields

* fix ParentVoterIDs

* fix verifier

* fix mock

* implement staking vote progressor

* fix tests

* move to module/packer

* fix tests

* fix validator test

* fix validator test

* use ParentVoterIndices

* add QuorumCertificateWithSignerIDs

* update blockSummary

* move module

* add test cases

* remove comments

* add check to EncodeSignerIndices

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* [Consensus] Use SignerIndices in CollectionGuarantee (3/4) (#2140)

* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
zhangchiqing added a commit that referenced this pull request Mar 30, 2022
* update interface

* remove IdentitiesByIndices

* remove TODO

* address review comments

* [Consensus] SignerIndices Optimization (2/4) (#2101)

* implement IdentitiesByIndices

* convert signer index

* implement encoding and decoding

* add test cases

* remove signer indices file

* fix Packer

* add comments

* remove IdentitiesByIndices

* update mock

* fix test cases

* add comments

* remove fields

* fix ParentVoterIDs

* fix verifier

* fix mock

* implement staking vote progressor

* fix tests

* move to module/packer

* fix tests

* fix validator test

* fix validator test

* use ParentVoterIndices

* add QuorumCertificateWithSignerIDs

* update blockSummary

* move module

* add test cases

* remove comments

* add check to EncodeSignerIndices

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* [Consensus] Use SignerIndices in CollectionGuarantee (3/4) (#2140)

* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
zhangchiqing added a commit that referenced this pull request Mar 30, 2022
* update interface

* remove IdentitiesByIndices

* remove TODO

* address review comments

* [Consensus] SignerIndices Optimization (2/4) (#2101)

* implement IdentitiesByIndices

* convert signer index

* implement encoding and decoding

* add test cases

* remove signer indices file

* fix Packer

* add comments

* remove IdentitiesByIndices

* update mock

* fix test cases

* add comments

* remove fields

* fix ParentVoterIDs

* fix verifier

* fix mock

* implement staking vote progressor

* fix tests

* move to module/packer

* fix tests

* fix validator test

* fix validator test

* use ParentVoterIndices

* add QuorumCertificateWithSignerIDs

* update blockSummary

* move module

* add test cases

* remove comments

* add check to EncodeSignerIndices

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* [Consensus] Use SignerIndices in CollectionGuarantee (3/4) (#2140)

* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
zhangchiqing added a commit that referenced this pull request Mar 30, 2022
* update interface

* remove IdentitiesByIndices

* remove TODO

* address review comments

* [Consensus] SignerIndices Optimization (2/4) (#2101)

* implement IdentitiesByIndices

* convert signer index

* implement encoding and decoding

* add test cases

* remove signer indices file

* fix Packer

* add comments

* remove IdentitiesByIndices

* update mock

* fix test cases

* add comments

* remove fields

* fix ParentVoterIDs

* fix verifier

* fix mock

* implement staking vote progressor

* fix tests

* move to module/packer

* fix tests

* fix validator test

* fix validator test

* use ParentVoterIndices

* add QuorumCertificateWithSignerIDs

* update blockSummary

* move module

* add test cases

* remove comments

* add check to EncodeSignerIndices

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* [Consensus] Use SignerIndices in CollectionGuarantee (3/4) (#2140)

* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
zhangchiqing added a commit that referenced this pull request May 25, 2022
* [Consensus] SignerIndices Optimization Interface changes (1/4) (#2047)

* update interface

* remove IdentitiesByIndices

* remove TODO

* address review comments

* [Consensus] SignerIndices Optimization (2/4) (#2101)

* implement IdentitiesByIndices

* convert signer index

* implement encoding and decoding

* add test cases

* remove signer indices file

* fix Packer

* add comments

* remove IdentitiesByIndices

* update mock

* fix test cases

* add comments

* remove fields

* fix ParentVoterIDs

* fix verifier

* fix mock

* implement staking vote progressor

* fix tests

* move to module/packer

* fix tests

* fix validator test

* fix validator test

* use ParentVoterIndices

* add QuorumCertificateWithSignerIDs

* update blockSummary

* move module

* add test cases

* remove comments

* add check to EncodeSignerIndices

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* Apply suggestions from code review

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* [Consensus] Use SignerIndices in CollectionGuarantee (3/4) (#2140)

* implement IdentitiesByIndices

* implement encoding and decoding

* remove IdentitiesByIndices

* use ParentVoterIndices

* use chainID

* add ChainID

* fix test case

* remove file

* sort assignment to canonical order

* fix guarantee

* fix execution ingestion core

* use signer indices

* add identifierOrder

* use guarantor.FindGuarantors

* fix tests

* fix tests

* fix tests

* disable tests

* fix tests

* fix tests, add validation

* fix fvm test

* fix access tests

* fix tests

* fix tests

* fix select_filter_test tests

* fix access tests

* fix access integration

* fix access integration tests

* revert fvm_test changes

* update fvm_test

* fix exec tests

* fix ingestion engine tests

* fix test cases

* remove todo

* fix error handling

* add comment

* fix tests

* update logging

* refactor

* fix tests

* remove function

* fix error wrapping

* update comment

* update tests

* remove unverified signature

* revert mutator tests change

* update comment

* address comments

* add tests

* add check in NewClusterList to ensure the assignments are sorted in
canonical order

* rename method

* consensus ingestion core log signer indices

* remove tests log

* refactor canonical order check

* replace order.ByNodeIDAsc with order.Canonical

* [Consensus] Refactor for guarantee signer indices (4/4) (#2204)

* wip

* wip

* starting to fix tests

* adding tests

* happy path test

* Added toDo for fixing tests for unhappy paths

* • fixed packer tests
• consolidated logic for checking the padded bits

* re-gen mocks

* fix validPadding

* fix findguarantors

* refactor ingestion/core.go

* move FindGuarantors to protocol

* remove commented code

* fix name

* fix tests

* small refactor

* fix import

* Apply suggestions from code review

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

* fix error type

* fix type

* fix error message

* update comments

* update comment

* update tests

* fix identity_test

* fix tests

* fix unittest

* fix cluster tests

* fix tests

* fixtures ingestion engine tests

* fix execution_test

* fix consensus inclusion tests

* fix bootstrap constraint check

* fix cycle dep from NewClusterList (#2225)

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>

* fix test cases

* update flow dep

* fix rpc convert

* update go.mod

* fix go.mod

* fix testnet network

* Remove unused travis yml

* update mocks

* upgrade flow

* update mock

* update go.sum

* fix hotstuff integration tests

* fix integration test

* first commit

* upgrade onflow/flow package to v0.3.0 (#2377)

* add checksum

* add checksum to signer indices

* fix execution tests

* fix execution tests

* adding comments

* fix ingestion tests

* fix access tests

* fix access tests

* fix access tests

* fix integration test

* add validation

* fix consensus integration tests

* fix flakey tests

* adding checks

* fix flakey tests

* fix check

* fix order

* fix sort

* use crc32 to compute checksum

* use canonical order

* fix qc_test

* Apply suggestions from code review

Co-authored-by: Tarak Ben Youssef <50252200+tarakby@users.noreply.github.com>

* update comments

* added BackendSCciptsMetrics interface and update noop/collector

* call the metrics

* switched to counter

* use BigEndian.PutUint32

* added buckets for script size

* go fmt

* added GetTransactionResultRTT metric

* record GetTransactionResultRTT metric

* record GetTransactionResultRTT metric

* check unit tests

* fix unit tests

* improve signer indices logging

* typo

* moved start time into loop

* go fmt

* remove util

* avoid race condition in validating blocks

* add snapshot command

* renamed methods and switched to histograms

* add comment on usage

* memory limit exception for service account

* fixed typos (#2443)

minor extension of error handling

* fixed imports

* address review comments

* lint

* fix comment

* update tests for validPadding

* refactor validPadding

* chore(docker): fix parallel image builds

* separated script size into two metrics

* payload_size label

* adding comments

* chore(tests): fix load test

Currently, test fails with the following error:
```
5:21PM ERR account creation tx failed error="[Error Code: 1101] cadence runtime error Execution failed:\nerror: invalid public key: [248, 71, 184, 64, 62, 118, 6, 35, 180, 178, 53, 126, 38, 116, 202, 182, 91, 243, 157, 205, 89, 77, 199, 235, 192, 117, 38, 122, 5, 166, 65, 116, 192, 67, 8, 190, 19, 100, 174, 246, 253, 218, 181, 121, 11, 79, 74, 202, 139, 156, 247, 215, 109, 152, 90, 162, 48, 116, 13, 180, 101, 48, 1, 40, 106, 13, 47, 226, 2, 3, 130, 3, 232]\n  --> 9d665e105234216749b51b3a301153c069b5c1047a1cf3f55e2ea38a5aa2c033:13:23\n   |\n13 |       let publicKey2 = PublicKey(\n   |                        ^\n"
```

It looks like it tries to pass the encoded public key (along w/ algo, hash, and weight) to a function that expects a raw public key.  Fix it by using the underlying PublicKey.

* unquarantining 2 flaky tests that have been consistently passing in quarantine (#2455)

* comments

* fix function renaming

* linter (#2457)

* Add memory weight for atree encoded slabs

* Add missing weights for cadence memory kinds

* Update to latest cadence

* Fix test

* [BFT Testing] Corruptible Conduit Factory authenticating dictated messages.  (#2441)

* makes generating execution receipt exportable

* (no branch): Auto stash before checking out "HEAD"

* removes corrupted execution result

* makes generating result approval exportable

* adds result approval generation to ccf

* fixes the tests

* adds attestation fixture

* adds message content as a parameter to message fixture

* fixes bug with publish

* wip

* adds test for result approval

* adds corrupted execution receipt test

* adds todos to factory

* refactors output types of wintermute orchestrator

* fixes lint issue

* removes unused variable

* Update insecure/wintermute/helpers.go

Co-authored-by: Misha <misha.rybalov@dapperlabs.com>

* Update insecure/wintermute/helpers.go

Co-authored-by: Misha <misha.rybalov@dapperlabs.com>

* fixes lint

* fixes receipt and approval bug with factory

* adds pass through test for receipt

* adds approval test

* fixes lint

* sorting imports

* sorting imports

Co-authored-by: Misha <misha.rybalov@dapperlabs.com>

* go mod tidy

* update Cadence

* update atree

Co-authored-by: Jordan Schalm <jordan@dapperlabs.com>
Co-authored-by: Alexander Hentschel <alex.hentschel@axiomzen.co>
Co-authored-by: Kay-Zee <kan@axiomzen.co>
Co-authored-by: Daniel  Holmes <danielholmes@dapper.local>
Co-authored-by: Tarak Ben Youssef <50252200+tarakby@users.noreply.github.com>
Co-authored-by: bors[bot] <26634292+bors[bot]@users.noreply.github.com>
Co-authored-by: danielholmes839 <danielh839@gmail.com>
Co-authored-by: Bastian Müller <bastian@axiomzen.co>
Co-authored-by: Daniel Holmes <43529937+danielholmes839@users.noreply.github.com>
Co-authored-by: Alexey Ivanov <SaveTheRbtz@GMail.com>
Co-authored-by: Misha <misha.rybalov@dapperlabs.com>
Co-authored-by: Supun Setunga <supun.setunga@gmail.com>
Co-authored-by: Yahya Hassanzadeh <yahya@dapperlabs.com>
Co-authored-by: Robert E. Davidson III <45945043+robert-e-davidson3@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants