Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adds ADR 038 - go plugin system #404

Merged
merged 73 commits into from
Feb 1, 2023

Conversation

egaxhaj
Copy link

@egaxhaj egaxhaj commented Dec 14, 2022

Implementation for ADR 038 - go plugin system

Description

Closes: #XXXX


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

SpicyLemon and others added 30 commits November 15, 2022 11:14
…m-change-proposal"

This reverts commit 7462a84, reversing
changes made to 23ba5dc.
* docs: add ApplicationQueryService release notes (cosmos#13587)

* docs: add ApplicationQueryService release notes

* updates

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(cli): add module-account cli cmd and grpc get api (backport cosmos#13612) (cosmos#13616)

* feat(cli): add module-account cli cmd and grpc get api (cosmos#13612)

(cherry picked from commit ddf5cf0)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/auth/v1beta1/query.pulsar.go
#	api/cosmos/auth/v1beta1/query_grpc.pb.go
#	x/auth/client/testutil/suite.go
#	x/auth/types/query.pb.go

* fix conflicts

* updates

* updates

Co-authored-by: Sai Kumar <17549398+gsk967@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (backport cosmos#12474) (cosmos#13598)

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (cosmos#12474)

(cherry picked from commit 18da0e9)

# Conflicts:
#	CHANGELOG.md

* fix changelog

* changelog

* fix: flakey test

Co-authored-by: Tyler <48813565+technicallyty@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: app-hash mismatch if upgrade migration commit is interrupted (backport cosmos#13530) (cosmos#13627)

* feat(cli): Add iavl-disable-fastnode cmd flag with proper description (cosmos#13656) (cosmos#13659)

(cherry picked from commit c833190)

Co-authored-by: William Chong <6198816+williamchong@users.noreply.github.com>

* build(deps): Bump github.com/cosmos/iavl from 0.19.3 to 0.19.4 (cosmos#13680)

* feat: emit cached context events (backport cosmos#13063) (cosmos#13702)

* chore: prepare 0.46.4 changelog (cosmos#13716)

* chore: prepare 0.46.4 changelog

* wording

* updates

* updates

* Change the default for the iavl-disable-fastnode flag to true (from false) to match the actual config default.

* Add changelog entry for the iavl fastnode flag default.

* Update swagger docs.

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <17549398+gsk967@users.noreply.github.com>
Co-authored-by: Tyler <48813565+technicallyty@users.noreply.github.com>
Co-authored-by: William Chong <6198816+williamchong@users.noreply.github.com>
* docs: add ApplicationQueryService release notes (cosmos#13587)

* docs: add ApplicationQueryService release notes

* updates

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(cli): add module-account cli cmd and grpc get api (backport cosmos#13612) (cosmos#13616)

* feat(cli): add module-account cli cmd and grpc get api (cosmos#13612)

(cherry picked from commit ddf5cf0)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/auth/v1beta1/query.pulsar.go
#	api/cosmos/auth/v1beta1/query_grpc.pb.go
#	x/auth/client/testutil/suite.go
#	x/auth/types/query.pb.go

* fix conflicts

* updates

* updates

Co-authored-by: Sai Kumar <17549398+gsk967@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (backport cosmos#12474) (cosmos#13598)

* fix(x/auth): allow multiple = signs in `GetTxsEvent` (cosmos#12474)

(cherry picked from commit 18da0e9)

# Conflicts:
#	CHANGELOG.md

* fix changelog

* changelog

* fix: flakey test

Co-authored-by: Tyler <48813565+technicallyty@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: app-hash mismatch if upgrade migration commit is interrupted (backport cosmos#13530) (cosmos#13627)

* feat(cli): Add iavl-disable-fastnode cmd flag with proper description (cosmos#13656) (cosmos#13659)

(cherry picked from commit c833190)

Co-authored-by: William Chong <6198816+williamchong@users.noreply.github.com>

* build(deps): Bump github.com/cosmos/iavl from 0.19.3 to 0.19.4 (cosmos#13680)

* feat: emit cached context events (backport cosmos#13063) (cosmos#13702)

* chore: prepare 0.46.4 changelog (cosmos#13716)

* chore: prepare 0.46.4 changelog

* wording

* updates

* updates

* ci: notify for 0.46.x releases (cosmos#13719)

* feat: notify for 0.46.x releases

* updates changelog

* docs: fix algolia on 0.46 (cosmos#13730)

* ci: modifying docs in 0.46 should not redeploy docs (cosmos#13732)

* ci: modifying docs in 0.46 should not redeploy docs

* updates

* fix: propagate msg events correctly in x/gov (backport cosmos#13728) (cosmos#13748)

* refactor: add error log when iavl set failed (backport cosmos#13803) (cosmos#13804)

* refactor: add error log when iavl set failed (cosmos#13803)

* add error log when iavl set failed

Ref: cosmos#12012

* Update CHANGELOG.md

* play safe

(cherry picked from commit 22f3261)

# Conflicts:
#	CHANGELOG.md

* fix conflicts

Co-authored-by: yihuang <huang@crypto.com>

* chore: bump tendermint to `0.34.23` (cosmos#13814)

* fix: propagate events in x/group through sdk.Results (backport cosmos#13808) (cosmos#13809)

* docs: update algolia index (cosmos#13823)

* fix: bank store migration (backport cosmos#13821) (cosmos#13829)

* fix: bank store migration (cosmos#13821)

(cherry picked from commit d314a12)

# Conflicts:
#	x/bank/migrations/v3/store.go
#	x/bank/migrations/v3/store_test.go

* updates

Co-authored-by: billy rennekamp <billy.rennekamp@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(types): set custom GasConfig on Context for GasKVStore (backport cosmos#13826) (cosmos#13833)

* feat(types): set custom GasConfig on Context for GasKVStore (cosmos#13826)

(cherry picked from commit f001b46)

# Conflicts:
#	CHANGELOG.md
#	types/context.go
#	types/context_test.go

* fix conflicts

* updates

Co-authored-by: Federico Kunze Küllmer <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* chore: prepare 0.46.5 release (cosmos#13816)

* fix: Allow underscores in EventRegex (backport cosmos#13861) (cosmos#13864)

* fix: Allow underscores in EventRegex (cosmos#13861)

(cherry picked from commit 14c582f)

# Conflicts:
#	CHANGELOG.md
#	tests/e2e/tx/service_test.go

* updates

* updates

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix(group): add group members weight checks (backport cosmos#13869) (cosmos#13880)

* fix(group): add group members weight checks (cosmos#13869)

(cherry picked from commit 3423442)

# Conflicts:
#	CHANGELOG.md
#	api/cosmos/tx/v1beta1/service.pulsar.go
#	types/tx/service.pb.go
#	x/group/keeper/keeper.go

* fix conflicts

* updates

* updates

Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix(group)!: Fix group min execution period (backport cosmos#13876) (cosmos#13885)

* fix(group)!: Fix group min execution period (cosmos#13876)

* fix: don't check MinExecutionPeriod in `Allow`

* Check MinExecutionPeriod on doExecuteMsgs

* Fix TestExec

* Fix TestExec

* test exec pruned

* Fix submitproposal

* Add changelog

* typo

* revert some changes

* add minExecutionPeriod

* Add docs and specs

(cherry picked from commit 7661f62)

# Conflicts:
#	CHANGELOG.md
#	x/group/README.md
#	x/group/keeper/keeper_test.go

* fix docs

* fix other conflicts

* fix test

Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(bank): Add helper for v0.46 denom migration (cosmos#13891)

* feat(bank): Add helper for v0.46 denom migration

* CL

* Clearer name

* Update x/bank/migrations/v046/store.go

Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>

* rename

Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>

* chore: impove 0.46.5 release notes (cosmos#13898)

* chore: impove 0.46.5 release notes

* update changelog

* refactor: State Streaming Docs + Explicit Config Support (backport cosmos#13894) (cosmos#13909)

* chore: remove typo (cosmos#13914) (cosmos#13917)

(cherry picked from commit 34dcca7)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: correctly propagate msg errors in gov (backport cosmos#13918) (cosmos#13928)

* fix: correctly propagate msg errors in gov (cosmos#13918)

* fix: correctly propagate msg errors in gov

* chore: update changelog

* fix: correctly check proposal status in tests

* chore: delete unused var

(cherry picked from commit 5581f7f)

# Conflicts:
#	CHANGELOG.md
#	x/gov/abci_test.go

* fix conflicts

* updates

Co-authored-by: John Letey <j@letey.de>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(bank): enable 0.46.5 bank migration fix through migrator keeper (cosmos#13929)

* Update changelog for version bump.

* Bring in Cosmos-SDK PR 13651 that overhauls/cleans up the GetConfig function to not require manual addition of stuff in order for it to be read.

* Fix the pruning command to use the existing viper instance (that has everything loaded into it) instead of creating a new one.

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Sai Kumar <17549398+gsk967@users.noreply.github.com>
Co-authored-by: Tyler <48813565+technicallyty@users.noreply.github.com>
Co-authored-by: William Chong <6198816+williamchong@users.noreply.github.com>
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: billy rennekamp <billy.rennekamp@gmail.com>
Co-authored-by: Federico Kunze Küllmer <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: Likhita Polavarapu <78951027+likhita-809@users.noreply.github.com>
Co-authored-by: John Letey <j@letey.de>
…smos#12648 (#368)

* feat: Add x/authz SendAuthorization AllowList (cosmos#12648)

Closes: cosmos#12609

---

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/main/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/main/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)

* Fix a unit test that for whatever reason didn't get updated in the merge.

Co-authored-by: likhita-809 <78951027+likhita-809@users.noreply.github.com>
and refactoring file streamer to output single file for each block

update spec

sort storeKey once at first

move MemoryListener store package
@egaxhaj egaxhaj force-pushed the prov/egaxhaj/v0.46.6-pio-1-add-adr038 branch from 7bd744f to db0557a Compare January 31, 2023 01:37
@SpicyLemon
Copy link

Running make test gives some compilation errors:

# github.com/cosmos/cosmos-sdk/x/group/internal/math [github.com/cosmos/cosmos-sdk/x/group/internal/math.test]
x/group/internal/math/dec_test.go:98:13: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/math/dec_test.go:99:7: invalid operation: rapid.Float64().Draw(t, "f") (value of type float64) is not an interface
x/group/internal/math/dec_test.go:112:21: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/math/dec_test.go:113:7: invalid operation: rapid.Float64().Draw(t, "f") (value of type float64) is not an interface
x/group/internal/math/dec_test.go:121:9: invalid operation: rapid.Int64().Draw(t, "n") (value of type int64) is not an interface
...
FAIL	github.com/cosmos/cosmos-sdk/x/group/internal/math [build failed]
# github.com/cosmos/cosmos-sdk/x/group/internal/orm [github.com/cosmos/cosmos-sdk/x/group/internal/orm.test]
x/group/internal/orm/primary_key_property_test.go:16:27: too many arguments in call to rapid.Run
	have (*primaryKeyMachine)
	want ()
x/group/internal/orm/primary_key_property_test.go:42:46: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/orm/primary_key_property_test.go:44:9: invalid operation: rapid.SampledFrom(m.stateKeys()).Draw(t, "key") (value of type string) is not an interface
x/group/internal/orm/generators_test.go:13:13: invalid operation: rapid.Uint64().Draw(t, "id") (value of type uint64) is not an interface
x/group/internal/orm/generators_test.go:14:13: invalid operation: rapid.StringN(1, 100, 150).Draw(t, "name") (value of type string) is not an interface
x/group/internal/orm/generators_test.go:15:13: invalid operation: rapid.Uint64().Draw(t, "number ") (value of type uint64) is not an interface
x/group/internal/orm/generators_test.go:16:20: invalid operation: rapid.StringN(1, 100, 150).Draw(t, "metadata") (value of type string) is not an interface
x/group/internal/orm/index_property_test.go:13:13: invalid operation: rapid.SliceOf(rapid.Byte()).Draw(t, "prefix") (value of type []byte) is not an interface
x/group/internal/orm/iterator_property_test.go:18:18: invalid operation: rapid.SliceOf(genTableModel).Draw(t, "tableModels") (value of type []*testdata.TableModel) is not an interface
x/group/internal/orm/iterator_property_test.go:25:12: invalid operation: rapid.Uint64Range(1, upperLimit).Draw(t, "limit") (value of type uint64) is not an interface
x/group/internal/orm/primary_key_property_test.go:44:9: too many errors
...
FAIL	github.com/cosmos/cosmos-sdk/x/group/internal/orm [build failed]

Our stuff is still based off of v0.46 which uses go 1.18. Using go 1.19 causes some tests to fail, so it shouldn't be used.

There's also lots of stuff that the SDK has added and/or overhauled in main that doesn't exist in v0.46. the /api and orm stuff are examples. So there's some stuff in this PR that shouldn't be.

server/config/config.go Outdated Show resolved Hide resolved
server/config/config.go Outdated Show resolved Hide resolved
server/config/toml.go Outdated Show resolved Hide resolved
server/config/toml.go Outdated Show resolved Hide resolved
simapp/app.go Outdated Show resolved Hide resolved
store/types/store.go Outdated Show resolved Hide resolved
server/config/config.go Outdated Show resolved Hide resolved
@egaxhaj
Copy link
Author

egaxhaj commented Jan 31, 2023

Running make test gives some compilation errors:

# github.com/cosmos/cosmos-sdk/x/group/internal/math [github.com/cosmos/cosmos-sdk/x/group/internal/math.test]
x/group/internal/math/dec_test.go:98:13: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/math/dec_test.go:99:7: invalid operation: rapid.Float64().Draw(t, "f") (value of type float64) is not an interface
x/group/internal/math/dec_test.go:112:21: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/math/dec_test.go:113:7: invalid operation: rapid.Float64().Draw(t, "f") (value of type float64) is not an interface
x/group/internal/math/dec_test.go:121:9: invalid operation: rapid.Int64().Draw(t, "n") (value of type int64) is not an interface
...
FAIL	github.com/cosmos/cosmos-sdk/x/group/internal/math [build failed]
# github.com/cosmos/cosmos-sdk/x/group/internal/orm [github.com/cosmos/cosmos-sdk/x/group/internal/orm.test]
x/group/internal/orm/primary_key_property_test.go:16:27: too many arguments in call to rapid.Run
 have (*primaryKeyMachine)
 want ()
x/group/internal/orm/primary_key_property_test.go:42:46: cannot use generic type rapid.Generator[V any] without instantiation
x/group/internal/orm/primary_key_property_test.go:44:9: invalid operation: rapid.SampledFrom(m.stateKeys()).Draw(t, "key") (value of type string) is not an interface
x/group/internal/orm/generators_test.go:13:13: invalid operation: rapid.Uint64().Draw(t, "id") (value of type uint64) is not an interface
x/group/internal/orm/generators_test.go:14:13: invalid operation: rapid.StringN(1, 100, 150).Draw(t, "name") (value of type string) is not an interface
x/group/internal/orm/generators_test.go:15:13: invalid operation: rapid.Uint64().Draw(t, "number ") (value of type uint64) is not an interface
x/group/internal/orm/generators_test.go:16:20: invalid operation: rapid.StringN(1, 100, 150).Draw(t, "metadata") (value of type string) is not an interface
x/group/internal/orm/index_property_test.go:13:13: invalid operation: rapid.SliceOf(rapid.Byte()).Draw(t, "prefix") (value of type []byte) is not an interface
x/group/internal/orm/iterator_property_test.go:18:18: invalid operation: rapid.SliceOf(genTableModel).Draw(t, "tableModels") (value of type []*testdata.TableModel) is not an interface
x/group/internal/orm/iterator_property_test.go:25:12: invalid operation: rapid.Uint64Range(1, upperLimit).Draw(t, "limit") (value of type uint64) is not an interface
x/group/internal/orm/primary_key_property_test.go:44:9: too many errors
...
FAIL	github.com/cosmos/cosmos-sdk/x/group/internal/orm [build failed]

Our stuff is still based off of v0.46 which uses go 1.18. Using go 1.19 causes some tests to fail, so it shouldn't be used.

There's also lots of stuff that the SDK has added and/or overhauled in main that doesn't exist in v0.46. the /api and orm stuff are examples. So there's some stuff in this PR that shouldn't be.

Could use a little help here. I don't know why these tests are failing.

The api/ folder contains only proto generated code for the plugin service after running make proto-gen task to generate the plugin service. As for the orm/, I don't see anything new being added from what's already in main-pio.

@SpicyLemon
Copy link

The api/ folder contains only proto generated code for the plugin service after running make proto-gen task to generate the plugin service. As for the orm/, I don't see anything new being added from what's already in main-pio.

Running make proto-gen in v0.46 does not create the /api stuff because it was added after v0.46. The reason it's in this PR is because you originally wrote the changes agains main and this is more of a backport, and you're erroneously including the files.

As for the compilation issues of those tests, my best guess is that one of the things you've bumped in the go.mod has a changed API causing the tests to not compile. Start by looking at the code that's failing to compile, and see what libraries are involved. If you really need the version bump on that library, then fix the tests, otherwise, don't bump that library.

server/config/toml.go Outdated Show resolved Hide resolved
server/config/toml.go Outdated Show resolved Hide resolved
@SpicyLemon
Copy link

The following commands run without error:

make test
export COSMOS_SDK_ABCI_V1="`pwd`/streaming/plugins/abci/v1/examples/file/file"
make test-sim-nondeterminism
make test-sim-nondeterminism
make test-sim-import-export
make test-sim-after-import
make test-sim-multi-seed-short

@SpicyLemon SpicyLemon merged commit 393db96 into main-pio Feb 1, 2023
@egaxhaj egaxhaj deleted the prov/egaxhaj/v0.46.6-pio-1-add-adr038 branch February 16, 2023 19:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants