Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate BaseAccount PubKey to use Any #7268

Merged
merged 183 commits into from
Sep 25, 2020
Merged

Migrate BaseAccount PubKey to use Any #7268

merged 183 commits into from
Sep 25, 2020

Conversation

blushi
Copy link
Contributor

@blushi blushi commented Sep 9, 2020

Description

ref: #6886 (comment)
closes: #6886

Also add back ed25519 support to the SDK:

  • create our own proto type ED25519
  • Inner Key field is []byte
  • Copy ed25519 files from Tendermint
  • implement crypto.PubKey on the proto struct

Workarounds added:

  • We register, in Amino, both TM's ed25519 and our own proto ed25519
  • Added a AsTmPubKey() method on the proto type, to convert our own ed25519 into TM's ed25519. Where TM expects a TM ed25519 (e.g. the tmtypes.NewValidator() function), we need to make sure to pass in pubkey.AsTmPubKey()
  • Added a workaround for bech32ifying ed25519 pubkeys without breaking backwards-compatibility.

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Targeted PR against correct branch (see CONTRIBUTING.md)
  • Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
  • Code follows the module structure standards.
  • Wrote unit and integration tests
  • Updated relevant documentation (docs/) or specification (x/<module>/spec/)
  • Added relevant godoc comments.
  • Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer
  • Review Codecov Report in the comment section below once CI passes

aaronc and others added 30 commits July 29, 2020 16:18
…ronc/revert-5682

# Conflicts:
#	testutil/testdata/proto.pb.go
client/context.go Show resolved Hide resolved
client/debug/main.go Outdated Show resolved Hide resolved
client/debug/main.go Outdated Show resolved Hide resolved
codec/yaml.go Show resolved Hide resolved
codec/yaml.go Outdated Show resolved Hide resolved
x/auth/types/account.go Show resolved Hide resolved
return pk
content, ok := acc.PubKey.GetCachedValue().(crypto.PubKey)
if !ok {
return nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we return nil here? Shouldn't we deserialize and set cache?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's a getter, how would you set the cached value? not sure to get what you mean.

if err != nil {
return nil, err
}

return string(bz), err
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Marshalers return []byte normally. Why do we convert to string?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why, removing it doesn't seem to cause any issue but I'd rather get other contributors's opinion on that first as it doesn't really relate to changes from this PR anyway.

Copy link
Contributor

@amaury1093 amaury1093 Sep 24, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I'm also not sure why. All MarshalYAML in the codebase return string, so at least it's consistent. Might be a follow-up PR to switch all to bytes, but it's a low-priority cleanup, I added it to #7357.

x/staking/types/validator.go Show resolved Hide resolved
x/staking/types/validator.go Outdated Show resolved Hide resolved
Copy link
Contributor

@clevinson clevinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Went through looking at code, not digging into the context though.

Few small suggestions / questions, otherwise lgtm!

client/debug/main.go Outdated Show resolved Hide resolved
client/context.go Show resolved Hide resolved
client/debug/main.go Outdated Show resolved Hide resolved
codec/yaml.go Show resolved Hide resolved
types/address.go Show resolved Hide resolved
types/address.go Outdated
Comment on lines 630 to 638
// This piece of code is to keep backwards-compatibility.
// For ed25519 keys, our own ed25519 is registered in Amino under a
// different name than TM's ed25519. But since users are already using
// TM's ed25519 bech32 encoding, we explicitly say to bech32-encode our own
// ed25519 the same way as TM's ed25519.
pkToMarshal := pubkey
if ed25519Pk, ok := pubkey.(*ed25519.PubKey); ok {
pkToMarshal = ed25519Pk.AsTmPubKey()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added suggested todo with link to issue.

x/auth/ante/sigverify.go Outdated Show resolved Hide resolved
blushi and others added 4 commits September 24, 2020 10:59
Co-authored-by: Cory <cjlevinson@gmail.com>
Co-authored-by: Cory <cjlevinson@gmail.com>
Copy link
Collaborator

@robert-zaremba robert-zaremba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving upfront, but let's sort out all comments and maybe create a new issues to followup.

x/staking/types/validator.go Outdated Show resolved Hide resolved
@blushi
Copy link
Contributor Author

blushi commented Sep 24, 2020

Approving upfront, but let's sort out all comments and maybe create a new issues to followup.

I believe @amaurymartiny added all required follow-ups in #7357.
Re: comments, doesn't this comment https://github.com/cosmos/cosmos-sdk/blob/marie/6928-baseacc-pk/x/staking/types/validator.go#L425-L429 sound okay to you?

@tac0turtle
Copy link
Member

Ah, this is bad I think. OK. Maybe we should sort out this dependencies later on.
-- what do you think? Why tendermint is doing this hard assumption for type, rather than using interfaces ? We should probably have a method and a separate interface for that, eg ProtoPubKeyExporter

In Tendermint we are currently only support ed25519 for this release. This was done in order to reduce the scope of work because we thought the release would be a few months ago. We will rework this part of Tendermint in the future to not have the type hardcoded.

As for the going to Tendermint key and sdk key, I am a bit scared of this design but it works for now. The dependency of Tendermint will change quite a bit in the coming releases.

Copy link
Contributor

@amaury1093 amaury1093 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this PR can be merged, follow-up work is tracked in #7357

@robert-zaremba if code comments are not clear in your opinion, feel free to push directly to this PR, it's better (as you said yesterday) than to go back and forth here.

@amaury1093
Copy link
Contributor

As for the going to Tendermint key and sdk key, I am a bit scared of this design

@marbar3778 Could you be a little bit more concrete why you are scared? Do you already have an idea of TM's refactor and how it'll impact the SDK having our own ed25519 like in this PR?

@tac0turtle
Copy link
Member

tac0turtle commented Sep 24, 2020

Could you be a little bit more concrete why you are scared? Do you already have an idea of TM's refactor and how it'll impact the SDK having our own ed25519 like in this PR?

Having to be aware of which key is being used and where. It is easy to miss something. The refactor of Tendermint will mainly hide more of the API so how things are done currently in places where Tendermint is used directly will need to be refactored to not use it.

@robert-zaremba
Copy link
Collaborator

@amaurymartiny , @marbar3778 -- exactly. This was my whole comment about. We should relay on interfaces and have good tests. We added followup checkpoint to solve this in the future.
Marko , if you have other things to add, please add it here: #7357

@amaury1093 amaury1093 added the A:automerge Automatically merge PR once all prerequisites pass. label Sep 25, 2020
@mergify mergify bot merged commit 91ca8ad into master Sep 25, 2020
@mergify mergify bot deleted the marie/6928-baseacc-pk branch September 25, 2020 08:41
larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023
* WIP on BaseAccount protobuf pub_key

* WIP on BaseAccount.PubKey

* WIP on BaseAccount pub key

* Update PubKey

* Update Account

* Docs

* WIP on protobuf keys

* Use Type() and Bytes() in sr25519 pub key Equals

* Add tests

* Add few more tests

* Update other pub/priv key types Equals

* Fix PrivKey's Sign method

* Rename variables in tests

* Fix infinite recursive calls

* Use tm ed25519 keys

* Add Sign and VerifySignature tests

* Remove ed25519 and sr25519 references

* proto linting

* Add proto crypto file

* Implement some of the new multisig proto type methods

* Add tests for MultisigThresholdPubKey

* Add tests for pubkey pb/amino conversion functions

* Move crypto types.go and register new proto pubkeys

* Add missing pointer ref

* Address review comments

* panic in MultisigThresholdPubKey VerifySignature

* Fix compile errors

* Remove pk conversion in ante handler

* Use internal crypto.PubKey in multisig

* Add tests for MultisigThresholdPubKey VerifyMultisignature

* Only keep LegacyAminoMultisigThresholdPubKey and move to proto keys to v1

* Remove conversion functions and introduce internal PubKey type

* Override Amino marshaling for proto pubkeys

* Merge master

* Make proto-gen

* Start removal of old PubKeyMultisigThreshold references

* Fix tests

* Fix solomachine

* Fix ante handler tests

* Pull latest go-amino

* Remove ed25519

* Remove old secp256k1 PubKey and PrivKey

* Uncomment test case

* Fix linting issues

* More linting

* Revert tests keys values

* Add Amino overrides to proto keys

* Add pubkey test

* Fix tests

* Use threshold isntead of K

* Standardize Type

* Revert standardize types commit

* Fix build

* Fix lint

* Fix lint

* Add comment

* Register crypto.PubKey

* Add empty key in BuildSimTx

* Simplify proto names

* Unpack interfaces for signing desc

* Fix IBC tests?

* Format proto

* Use secp256k1 in ibc

* Fixed merge issues

* Uncomment tests

* Update x/ibc/testing/solomachine.go

* UnpackInterfaces for solomachine types

* Remove old multisig

* Add amino marshal for multisig

* Fix lint

* Correctly register amino

* One test left!

* Remove old struct

* Fix test

* Fix test

* Unpack into tmcrypto

* Remove old threshold pubkey tests

* Fix register amino

* Fix lint

* Use sdk crypto PubKey in multisig UnpackInterfaces

* Potential fix?

* Register LegacyAminoPubKey

* Register our own PubKey

* Register tmcrypto PubKey

* Register both PubKeys

* Register interfaces in test

* Refactor fiels

* Add comments

* Remove old cosmos-sdk/crypto/keys reference

* Use anil's suggestion

* Add norace back

* Use our own ed25519

* Fix pubkey types

* Fix network tests

* Fix more tests

* Make ibc work?

* Use TM pubkey in NewValidator

* Fix lint

* Put interface in tpyes

* rerun CI

* Better name register

* Remove stray code

* Add ed25519 tests

* Check nil

* Correct interface impl assert

* rerun CI

* Add fix for Bech32

* Address comments

* FIx lint

* Add tests for solomachine unpack interfaces

* Fix query tx by hash

* Better name in amino register

* Fix lint

* Add back ed25519 test (fixes cosmos#7352)

* go mod tidy

* Fix merge issues

* Sort import

* Add test for backwards-compatibility

* Fix tests

* Fix merge issue

* Update client/context.go

Co-authored-by: Cory <cjlevinson@gmail.com>

* Update types/address.go

Co-authored-by: Cory <cjlevinson@gmail.com>

* Address feedback

* Add comment

* Fix BaseAccount SetPubKey and address further comments

* Lint

* Remove unnecessary use of copy in getPubKeyFromString

* Update comment

Co-authored-by: Aaron Craelius <aaronc@users.noreply.github.com>
Co-authored-by: Aaron Craelius <aaron@regen.network>
Co-authored-by: Amaury Martiny <amaury.martiny@protonmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: colin axnér <25233464+colin-axner@users.noreply.github.com>
Co-authored-by: Cory <cjlevinson@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A:automerge Automatically merge PR once all prerequisites pass.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use protobuf public key in BaseAccount
8 participants