Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add local snapshots management commands #16067

Merged
merged 15 commits into from
May 11, 2023

Conversation

yihuang
Copy link
Collaborator

@yihuang yihuang commented May 9, 2023

Closes: #16066

Manage local snapshots

Usage:
  simd snapshots [command]

Available Commands:
  delete      Delete a local snapshot
  dump        Dump the snapshot as portable archive format
  export      Export app state to snapshot store
  list        List local snapshots
  load        Load a snapshot archive file into snapshot store
  restore     Restore app state from local snapshot

Flags:
  -h, --help   help for snapshots

Global Flags:
      --home string         directory for config and data (default "/Users/yihuang/.simapp")
      --log_format string   The logging format (json|plain) (default "plain")
      --log_level string    The logging level (trace|debug|info|warn|error|fatal|panic) (default "info")
      --trace               print out full stack trace on errors

Use "simd snapshots [command] --help" for more information about a command.

Description

Closes: #XXXX


Author Checklist

All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.

I have...

  • included the correct type prefix in the PR title
  • added ! to the type prefix if API or client breaking change
  • targeted the correct branch (see PR Targeting)
  • provided a link to the relevant issue or specification
  • followed the guidelines for building modules
  • included the necessary unit and integration tests
  • added a changelog entry to CHANGELOG.md
  • included comments for documenting Go code
  • updated the relevant documentation or specification
  • reviewed "Files changed" and left comments if necessary
  • confirmed all CI checks have passed

Reviewers Checklist

All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.

I have...

  • confirmed the correct type prefix in the PR title
  • confirmed ! in the type prefix if API or client breaking change
  • confirmed all author checklist items have been addressed
  • reviewed state machine logic
  • reviewed API design and naming
  • reviewed documentation is accurate
  • reviewed tests and test coverage
  • manually tested (if applicable)

@yihuang yihuang requested a review from a team as a code owner May 9, 2023 03:19
@github-actions github-actions bot added the C:CLI label May 9, 2023
- `appd snapshots list`
  list local snapshots
- `appd snapshots restore`
  restore from a local snapshot
CHANGELOG.md Outdated Show resolved Hide resolved
Comment on lines 418 to +424
return false, nil
}

// RestoreLocalSnapshot restores app state from a local snapshot.
func (m *Manager) RestoreLocalSnapshot(height uint64, format uint32) error {
snapshot, ch, err := m.store.Load(height, format)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change potentially affects state.

Call sequence:

(*cosmossdk.io/store/snapshots.Manager).sortedExtensionNames (store/snapshots/manager.go:422)
(*cosmossdk.io/store/snapshots.Manager).createSnapshot (store/snapshots/manager.go:192)
(*cosmossdk.io/store/snapshots.Manager).Create (store/snapshots/manager.go:161)
(*cosmossdk.io/store/snapshots.Manager).snapshot (store/snapshots/manager.go:460)
(*cosmossdk.io/store/snapshots.Manager).SnapshotIfApplicable (store/snapshots/manager.go:444)
(*github.com/cosmos/cosmos-sdk/baseapp.BaseApp).Commit (store/snapshots/manager.go:458)

Copy link
Member

@julienrbrt julienrbrt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice! utACK.

@yihuang
Copy link
Collaborator Author

yihuang commented May 9, 2023

I'll add another command to make portable snapshots, basically bundle the Snapshot message together with the chunk files, which can be exchanged outside of the p2p network.

Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK

@yihuang
Copy link
Collaborator Author

yihuang commented May 10, 2023

More complete now:

$ ./build/simd snapshots --help
WARNING: proto: file "google/protobuf/any.proto" already registered
Manage local snapshots

Usage:
  simd snapshots [command]

Available Commands:
  delete      Delete a local snapshot
  dump        Dump the snapshot for portable archive format
  list        List snapshots
  load        Load a snapshot archive file into snapshot store
  restore     Restore app state from local snapshot

Flags:
  -h, --help   help for snapshots

Global Flags:
      --home string         directory for config and data (default "/Users/yihuang/.simapp")
      --log_format string   The logging format (json|plain) (default "plain")
      --log_level string    The logging level (trace|debug|info|warn|error|fatal|panic) (default "info")
      --trace               print out full stack trace on errors

Use "simd snapshots [command] --help" for more information about a command.

Example for dump/load:

$ ./build/simd snapshots dump 8190000 2 --home /tmp/test1 --output /tmp/arhive-8190000-2.tar.gz
$ ./build/simd snapshots load /tmp/arhive-8190000-2.tar.gz --home /tmp/test2
$ ./build/simd snapshots list --home /tmp/test2
height: 8190000 format: 2 chunks: 270

client/snapshot/dump.go Outdated Show resolved Hide resolved
client/snapshot/dump.go Outdated Show resolved Hide resolved
client/snapshot/dump.go Outdated Show resolved Hide resolved
client/snapshot/list.go Outdated Show resolved Hide resolved
}

path := args[0]
fp, err := os.Open(path)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable

for i := uint32(0); i < snapshot.Chunks; i++ {
path := snapshotStore.PathChunk(height, uint32(format), i)
file, err := os.Open(path)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
return err
}

fp, err := os.Create(output)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
if err != nil {
return err
}
defer fp.Close()

Check failure

Code scanning / gosec

Deferring unsafe method "Close" on type "net.Listener"

Deferring unsafe method "Close" on type "*os.File"
"io"
"io/ioutil"
"os"
"reflect"

Check notice

Code scanning / CodeQL

Sensitive package import

Certain system packages contain functions which may be a possible source of non-determinism
Comment on lines +69 to +78
go func() {
defer close(quitChan)

savedSnapshot, err := snapshotStore.Save(snapshot.Height, snapshot.Format, chunks)
if err != nil {
fmt.Println("failed to save snapshot", err)
return
}
quitChan <- savedSnapshot
}()

Check notice

Code scanning / CodeQL

Spawning a Go routine

Spawning a Go routine may be a possible source of non-determinism
@tac0turtle tac0turtle added backport/0.46.x PR scheduled for inclusion in the v0.46's next stable release backport/v0.47.x PR scheduled for inclusion in the v0.47's next stable release labels May 10, 2023
Copy link
Member

@tac0turtle tac0turtle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets fix the linting then we are good to merge

@tac0turtle tac0turtle added this pull request to the merge queue May 11, 2023
Merged via the queue into cosmos:main with commit c1ceb3b May 11, 2023
mergify bot pushed a commit that referenced this pull request May 11, 2023
Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit c1ceb3b)

# Conflicts:
#	go.mod
#	go.sum
#	server/types/app.go
#	server/util.go
#	simapp/simd/cmd/root.go
#	simapp/simd/cmd/root_v2.go
#	snapshots/store.go
mergify bot pushed a commit that referenced this pull request May 11, 2023
Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit c1ceb3b)

# Conflicts:
#	CHANGELOG.md
#	go.mod
#	server/types/app.go
#	simapp/simd/cmd/root.go
#	simapp/simd/cmd/root_v2.go
#	snapshots/store.go
@yihuang yihuang deleted the snapshot-cmds branch May 11, 2023 10:35
kocubinski pushed a commit that referenced this pull request May 11, 2023
Co-authored-by: Marko <marbar3778@yahoo.com>
julienrbrt added a commit that referenced this pull request May 12, 2023
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
julienrbrt added a commit that referenced this pull request May 12, 2023
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
larry0x pushed a commit to larry0x/cosmos-sdk that referenced this pull request May 22, 2023
Co-authored-by: Marko <marbar3778@yahoo.com>
@chillyvee
Copy link
Contributor

@yihuang - very useful

  1. Any plans to backport to earlier cosmos-sdk versions? Should we try to assist?
  2. Which testnet/mainnet chain did you test this code against? (if any)

@yihuang
Copy link
Collaborator Author

yihuang commented May 22, 2023

@yihuang - very useful

  1. Any plans to backport to earlier cosmos-sdk versions? Should we try to assist?

  2. Which testnet/mainnet chain did you test this code against? (if any)

Has back-ported to 0.47 0.46, we plan to release later

SpicyLemon added a commit to provenance-io/cosmos-sdk that referenced this pull request Jun 9, 2023
* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065)

Co-authored-by: yihuang <huang@crypto.com>

* chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115)

Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* refactor!: remove store v2alpha1 (cosmos#15152)

* build: use cometbft in v0.46 (cosmos#15090)

* chore: bump cometbft to v0.34.27 final (cosmos#15206)

* fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245)

Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: revert double close fix (cosmos#15255)

* chore: prepare v0.46.11 release (cosmos#15259)

* feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321)

* feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349)

* fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* docs: update cosmwasm link (cosmos#15441)

* fix: remove extra `;` in service.proto (cosmos#15443)

* fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461)

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473)

Co-authored-by: Marko <marbar3778@yahoo.com>

* docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476)

Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525)

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)

Co-authored-by: Daniel Wedul <github@wedul.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

* fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669)

Co-authored-by: Javier Su <javier.su.weijie@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* chore: prepare v0.46.12 release notes (cosmos#15685)

* fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717)

Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775)

Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925)

Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* build(deps): bump cometbft to v0.34.28 (cosmos#15973)

* chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981)

Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* feat!: bootstrap comet cmd for local state sync  (backport cosmos#16061) (cosmos#16080)

Co-authored-by: Marko <marbar3778@yahoo.com>

* chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* chore: update sidebar v0.46 (cosmos#16304)

* chore: add baseapp circuit breaker setter (cosmos#16289)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306)

Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331)

* chore: prepare v0.46.13 (cosmos#16249)

* chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* Add the quarantine and sanction stores to the import-export sim test.

* refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432)

Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Facundo Medica <facundomedica@gmail.com>

* chore: prepare v0.46.13 (2/2) (cosmos#16443)

* fix: patch barberry (cosmos#16466)

* Better gov prop error handling.

* Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do.

* Add changelog entries.

* Regen the swagger docs.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Javier Su <javier.su.weijie@gmail.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com>
Co-authored-by: Facundo Medica <facundomedica@gmail.com>
SpicyLemon added a commit to provenance-io/cosmos-sdk that referenced this pull request Jun 9, 2023
* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065)

Co-authored-by: yihuang <huang@crypto.com>

* chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115)

Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* refactor!: remove store v2alpha1 (cosmos#15152)

* build: use cometbft in v0.46 (cosmos#15090)

* chore: bump cometbft to v0.34.27 final (cosmos#15206)

* fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245)

Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: revert double close fix (cosmos#15255)

* chore: prepare v0.46.11 release (cosmos#15259)

* feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321)

* feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349)

* fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* docs: update cosmwasm link (cosmos#15441)

* fix: remove extra `;` in service.proto (cosmos#15443)

* fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461)

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473)

Co-authored-by: Marko <marbar3778@yahoo.com>

* docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476)

Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525)

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)

Co-authored-by: Daniel Wedul <github@wedul.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

* fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669)

Co-authored-by: Javier Su <javier.su.weijie@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* chore: prepare v0.46.12 release notes (cosmos#15685)

* fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717)

Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775)

Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925)

Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* build(deps): bump cometbft to v0.34.28 (cosmos#15973)

* chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981)

Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* feat!: bootstrap comet cmd for local state sync  (backport cosmos#16061) (cosmos#16080)

Co-authored-by: Marko <marbar3778@yahoo.com>

* chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>

* chore: update sidebar v0.46 (cosmos#16304)

* chore: add baseapp circuit breaker setter (cosmos#16289)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306)

Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>

* fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331)

* chore: prepare v0.46.13 (cosmos#16249)

* chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409)

Co-authored-by: Julien Robert <julien@rbrt.fr>

* Add the quarantine and sanction stores to the import-export sim test.

* refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432)

Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Facundo Medica <facundomedica@gmail.com>

* chore: prepare v0.46.13 (2/2) (cosmos#16443)

* fix: patch barberry (cosmos#16466)

* Better gov prop error handling.

* Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do.

* Add changelog entries.

* Regen the swagger docs.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Javier Su <javier.su.weijie@gmail.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com>
Co-authored-by: Facundo Medica <facundomedica@gmail.com>
SpicyLemon added a commit to provenance-io/cosmos-sdk that referenced this pull request Jun 9, 2023
* fix: create snapshot dir for rocksdb backend (backport cosmos#15054) (cosmos#15065)



* chore: Add check for uneven stores' height (backport cosmos#14410) (cosmos#15115)




* refactor!: remove store v2alpha1 (cosmos#15152)

* build: use cometbft in v0.46 (cosmos#15090)

* chore: bump cometbft to v0.34.27 final (cosmos#15206)

* fix: Change proposer address cast for `sdk_block` conversion (backport cosmos#15243) (cosmos#15245)




* fix: revert double close fix (cosmos#15255)

* chore: prepare v0.46.11 release (cosmos#15259)

* feat: add extend cb to avoid unmarshal state twice for sim test (backport: cosmos#15305) (cosmos#15321)

* feat: add extend cb with genesisState for sim test (backport: cosmos#15305) (cosmos#15349)

* fix: add extra check in vesting (backport cosmos#15373) (cosmos#15383)



* docs: update cosmwasm link (cosmos#15441)

* fix: remove extra `;` in service.proto (cosmos#15443)

* fix: flaky group genesis sim (backport cosmos#15447) (cosmos#15461)



* feat: add delegator to withdraw address (backport cosmos#15462) (cosmos#15473)



* docs: improve proto vesting docs about time attributes (backport cosmos#15474) (cosmos#15481)



* feat: add query `groups` in `x/group` (backport cosmos#14879) (cosmos#15476)




* fix: Makefile rocksdb (backport cosmos#15422) (cosmos#15525)



* fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)




* fix: remove unnecessary cms typecasting (backport cosmos#14054) (cosmos#15669)




* chore: prepare v0.46.12 release notes (cosmos#15685)

* fix: upstream error on empty version (backport cosmos#13355) (cosmos#15717)




* feat: More flexibility for `CacheMultiStoreWithVersion` (backport cosmos#15683) (cosmos#15775)




* feat: add moduleStateCb to allow access moduleState in sim test (backport cosmos#15903) (cosmos#15925)




* build(deps): bump cometbft to v0.34.28 (cosmos#15973)

* chore(gov): improve proposal conversion error message (backport cosmos#15979) (cosmos#15981)




* feat!: bootstrap comet cmd for local state sync  (backport cosmos#16061) (cosmos#16080)



* chore(auth/vesting): fix typo in `create-period-vesting-account` cmd example (backport cosmos#16085) (cosmos#16087)



* feat: add local snapshots management commands (backport cosmos#16067) (cosmos#16103)





* fix: snapshot commands panic if snapshot don't exists (backport cosmos#16138) (cosmos#16140)





* feat: add Close method for resource cleanup in graceful shutdown (backport cosmos#16193) (cosmos#16205)




* feat: save restored snapshot locally (backport cosmos#16060) (cosmos#16262)




* chore: update sidebar v0.46 (cosmos#16304)

* chore: add baseapp circuit breaker setter (cosmos#16289)



* fix: allow any address in `ValidatePromptAddress` (backport cosmos#16312) (cosmos#16314)



* feat(group): add group event tally result (backport cosmos#16191) (cosmos#16306)





* fix: Revert "fix(x/gov): Return ErrInvalidProposalContent in SubmitProposal when legacy handler returns an error. (backport cosmos#13051) (cosmos#15667)" (cosmos#16331)

* chore: prepare v0.46.13 (cosmos#16249)

* chore: small snapshot commands & docs improvement (backport cosmos#16404) (cosmos#16409)



* Add the quarantine and sanction stores to the import-export sim test.

* refactor: avoid breaking change due to cosmos#16415 included in v0.50 (backport cosmos#16430) (cosmos#16432)




* chore: prepare v0.46.13 (2/2) (cosmos#16443)

* fix: patch barberry (cosmos#16466)

* Better gov prop error handling.

* Fix the group sims random genesis so it can't pick the same account twice for the group policies since that address is the primary key for the table. Also, output the genesis state after it's randomized just like all the other modules do.

* Add changelog entries.

* Regen the swagger docs.

---------

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: khanh-notional <50263489+catShaark@users.noreply.github.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Co-authored-by: Maksym Hontar <99190705+max-hontar@users.noreply.github.com>
Co-authored-by: mmsqe <mavis@crypto.com>
Co-authored-by: Robert Zaremba <robert@zaremba.ch>
Co-authored-by: atheeshp <59333759+atheeshp@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Javier Su <javier.su.weijie@gmail.com>
Co-authored-by: mmsqe <tqd0800210105@gmail.com>
Co-authored-by: Jeancarlo Barrios <JeancarloBarrios@users.noreply.github.com>
Co-authored-by: Facundo Medica <facundomedica@gmail.com>
roy-dydx pushed a commit to dydxprotocol/cosmos-sdk that referenced this pull request Jul 11, 2023
…cosmos#16102)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
kakysha pushed a commit to InjectiveLabs/cosmos-sdk that referenced this pull request Aug 15, 2023
…cosmos#16102)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
@holyxiaoxin
Copy link

holyxiaoxin commented Apr 18, 2024

My nodes are too bloated and they stopped producing snapshots, and I found these snapshots management commands hoping that I could export a node's snapshot and restore this snapshot into a newly init chain on another node.

@yihuang What is the workflow to restore the snapshot from a portable archive? I ran appd snapshots export to export latest state to snapshot store. Then ran appd snapshots dump <height> 3 to dump archive file. Then on a newly app init node with cofigurations set properly, with chain's config/genesis.json (state-syncable), I ran appd snapshots load height-3.tar.gz. Then ran appd snapshots restore 5 3 to restore chain state to snapshot. But after starting the chain, I received the following error:

panic: state.AppHash does not match AppHash after replay. Got
38C37A1E4FD2D3376663C6C87CD91660F7E5B432BBDDDB97939776D43489B510, expected .
State: {{{11 0} 0.37.2} app 1 0 :0:000000000000 2024-04-18 05:45:49.278222786 +0000 UTC ValidatorSet{
  Proposer: nil-Validator
  Validators:
    
} ValidatorSet{
  Proposer: nil-Validator
  Validators:
    
} ValidatorSet{
  Proposer: nil-Validator
  Validators:
    
} 1 {{22020096 -1} {100000 48h0m0s 1048576} {[ed25519]} {0}} 1 [] []}

Did you reset CometBFT without resetting your application's data? [recovered]
	panic: state.AppHash does not match AppHash after replay. Got
38C37A1E4FD2D3376663C6C87CD91660F7E5B432BBDDDB97939776D43489B510, expected .

Did you reset CometBFT without resetting your application's data?

I've also tried removing all the db folders before restoring, but got the same error.

When trying to do a snapshot statesync, I received the following error when starting the chain:

ERR failed to restore snapshot err="multistore restore: import failed: found database at version 56746000, must be 0" module=server
ERR State sync failed err="state sync aborted" module=statesync server=node

@holyxiaoxin holyxiaoxin mentioned this pull request Apr 29, 2024
20 tasks
JeancarloBarrios pushed a commit to agoric-labs/cosmos-sdk that referenced this pull request Sep 28, 2024
…cosmos#16103)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/v0.47.x PR scheduled for inclusion in the v0.47's next stable release backport/0.46.x PR scheduled for inclusion in the v0.46's next stable release C:CLI C:Store
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature]: use plain files to store snapshots
5 participants