feat: add local snapshots management commands (backport #16067) #16103

mergify · 2023-05-11T10:15:49Z

This is an automatic backport of pull request #16067 done by Mergify.
Cherry-pick of c1ceb3b has failed:

On branch mergify/bp/release/v0.46.x/pr-16067
Your branch is up to date with 'origin/release/v0.46.x'.

You are currently cherry-picking commit c1ceb3bdd.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   CHANGELOG.md
	new file:   client/snapshot/cmd.go
	new file:   client/snapshot/delete.go
	new file:   client/snapshot/dump.go
	new file:   client/snapshot/export.go
	new file:   client/snapshot/list.go
	new file:   client/snapshot/load.go
	new file:   client/snapshot/restore.go
	modified:   snapshots/manager.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   go.mod
	both modified:   go.sum
	both modified:   server/types/app.go
	both modified:   server/util.go
	both modified:   simapp/simd/cmd/root.go
	deleted by us:   simapp/simd/cmd/root_v2.go
	both modified:   snapshots/store.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

@Mergifyio refresh will re-evaluate the rules
@Mergifyio rebase will rebase this PR on its base branch
@Mergifyio update will merge the base branch into this PR
@Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

look at your merge queues
generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit c1ceb3b) # Conflicts: # go.mod # go.sum # server/types/app.go # server/util.go # simapp/simd/cmd/root.go # simapp/simd/cmd/root_v2.go # snapshots/store.go

client/snapshot/load.go

+			go func() {
+				defer close(quitChan)
+
+				savedSnapshot, err := snapshotStore.Save(snapshot.Height, snapshot.Format, chunks)
+				if err != nil {
+					fmt.Println("failed to save snapshot", err)
+					return
+				}
+				quitChan <- savedSnapshot
+			}()


client/snapshot/load.go

+	"fmt"
+	"io"
+	"os"
+	"reflect"


server/types/app.go

tac0turtle · 2023-05-12T09:12:41Z

not sure why ci is not running here? @julienrbrt any ideas?

julienrbrt · 2023-05-12T09:16:35Z

not sure why ci is not running here? @julienrbrt any ideas?

I think because of GH issues that happened at around the time of the last commit? I had to re-trigger a PR by submitting a commit yesterday because of that.

Co-authored-by: yihuang <huang@crypto.com>

* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065) Co-authored-by: yihuang <huang@crypto.com> * chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115) Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * refactor!: remove store v2alpha1 (cosmos#15152) * build: use cometbft in v0.46 (cosmos#15090) * chore: bump cometbft to v0.34.27 final (cosmos#15206) * fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245) Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: revert double close fix (cosmos#15255) * chore: prepare v0.46.11 release (cosmos#15259) * feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321) * feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349) * fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383) Co-authored-by: Julien Robert <julien@rbrt.fr> * docs: update cosmwasm link (cosmos#15441) * fix: remove extra `;` in service.proto (cosmos#15443) * fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461) Co-authored-by: Robert Zaremba <robert@zaremba.ch> * feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473) Co-authored-by: Marko <marbar3778@yahoo.com> * docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476) Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525) Co-authored-by: Robert Zaremba <robert@zaremba.ch> * fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667) Co-authored-by: Daniel Wedul <github@wedul.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> * fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669) Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * chore: prepare v0.46.12 release notes (cosmos#15685) * fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717) Co-authored-by: Marko <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775) Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925) Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * build(deps): bump cometbft to v0.34.28 (cosmos#15973) * chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981) Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * feat!: bootstrap comet cmd for local state sync (backport cosmos#16061) (cosmos#16080) Co-authored-by: Marko <marbar3778@yahoo.com> * chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> * chore: update sidebar v0.46 (cosmos#16304) * chore: add baseapp circuit breaker setter (cosmos#16289) Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314) Co-authored-by: Julien Robert <julien@rbrt.fr> * feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306) Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> * fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331) * chore: prepare v0.46.13 (cosmos#16249) * chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409) Co-authored-by: Julien Robert <julien@rbrt.fr> * Add the quarantine and sanction stores to the import-export sim test. * refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432) Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Facundo Medica <facundomedica@gmail.com> * chore: prepare v0.46.13 (2/2) (cosmos#16443) * fix: patch barberry (cosmos#16466) * Better gov prop error handling. * Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do. * Add changelog entries. * Regen the swagger docs. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Robert Zaremba <robert@zaremba.ch> Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: Facundo Medica <facundomedica@gmail.com>

* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065) * chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115) * refactor!: remove store v2alpha1 (cosmos#15152) * build: use cometbft in v0.46 (cosmos#15090) * chore: bump cometbft to v0.34.27 final (cosmos#15206) * fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245) * fix: revert double close fix (cosmos#15255) * chore: prepare v0.46.11 release (cosmos#15259) * feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321) * feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349) * fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383) * docs: update cosmwasm link (cosmos#15441) * fix: remove extra `;` in service.proto (cosmos#15443) * fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461) * feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473) * docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481) * feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476) * fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525) * fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667) * fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669) * chore: prepare v0.46.12 release notes (cosmos#15685) * fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717) * feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775) * feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925) * build(deps): bump cometbft to v0.34.28 (cosmos#15973) * chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981) * feat!: bootstrap comet cmd for local state sync (backport cosmos#16061) (cosmos#16080) * chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087) * feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103) * fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140) * feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205) * feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262) * chore: update sidebar v0.46 (cosmos#16304) * chore: add baseapp circuit breaker setter (cosmos#16289) * fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314) * feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306) * fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331) * chore: prepare v0.46.13 (cosmos#16249) * chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409) * Add the quarantine and sanction stores to the import-export sim test. * refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432) * chore: prepare v0.46.13 (2/2) (cosmos#16443) * fix: patch barberry (cosmos#16466) * Better gov prop error handling. * Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do. * Add changelog entries. * Regen the swagger docs. --------- Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr> Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com> Co-authored-by: mmsqe <mavis@crypto.com> Co-authored-by: Robert Zaremba <robert@zaremba.ch> Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com> Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com> Co-authored-by: Javier Su <javier.su.weijie@gmail.com> Co-authored-by: mmsqe <tqd0800210105@gmail.com> Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com> Co-authored-by: Facundo Medica <facundomedica@gmail.com>

…cosmos#16103) Co-authored-by: yihuang <huang@crypto.com> Co-authored-by: marbar3778 <marbar3778@yahoo.com> Co-authored-by: Julien Robert <julien@rbrt.fr>

feat: add local snapshots management commands (#16067)

d46d7cb

Co-authored-by: Marko <marbar3778@yahoo.com> (cherry picked from commit c1ceb3b) # Conflicts: # go.mod # go.sum # server/types/app.go # server/util.go # simapp/simd/cmd/root.go # simapp/simd/cmd/root_v2.go # snapshots/store.go

mergify bot requested a review from a team as a code owner May 11, 2023 10:15

mergify bot added the conflicts label May 11, 2023

resolve conflicts

fb8b50e

github-actions bot added the C:CLI label May 11, 2023

tac0turtle removed the conflicts label May 11, 2023

github-advanced-security bot found potential problems May 11, 2023

View reviewed changes

julienrbrt and others added 5 commits May 11, 2023 13:54

add missing defer (#16107)

bf2939a

fix linting

734f2da

bring back save chunk

12af89a

fix imports

ab247e6

fix import

348be87

tac0turtle approved these changes May 11, 2023

View reviewed changes

yihuang reviewed May 12, 2023

View reviewed changes

server/types/app.go Outdated Show resolved Hide resolved

yihuang approved these changes May 12, 2023

View reviewed changes

julienrbrt and others added 5 commits May 12, 2023 11:19

retrigger

0d9068f

Co-authored-by: yihuang <huang@crypto.com>

fix GHSA-3vp4-m3rf-835h

03d6684

updates

8002a68

downgrade goleveldb

27075aa

make rosetta-data

8d62335

github-actions bot added the C:Rosetta Issues and PR related to Rosetta label May 12, 2023

julienrbrt enabled auto-merge (squash) May 12, 2023 10:09

updates

5e9d750

julienrbrt merged commit 10cc30f into release/v0.46.x May 12, 2023

julienrbrt deleted the mergify/bp/release/v0.46.x/pr-16067 branch May 12, 2023 10:22

faddat mentioned this pull request Jun 8, 2023

barberry realiotech/cosmos-sdk#30

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add local snapshots management commands (backport #16067) #16103

feat: add local snapshots management commands (backport #16067) #16103

mergify bot commented May 11, 2023

tac0turtle commented May 12, 2023

julienrbrt commented May 12, 2023

feat: add local snapshots management commands (backport #16067) #16103

feat: add local snapshots management commands (backport #16067) #16103

Conversation

mergify bot commented May 11, 2023

tac0turtle commented May 12, 2023

julienrbrt commented May 12, 2023