Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add local snapshots management commands (backport #16067) #16102

Merged
merged 11 commits into from
May 12, 2023

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented May 11, 2023

This is an automatic backport of pull request #16067 done by Mergify.
Cherry-pick of c1ceb3b has failed:

On branch mergify/bp/release/v0.47.x/pr-16067
Your branch is up to date with 'origin/release/v0.47.x'.

You are currently cherry-picking commit c1ceb3bdd.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	new file:   client/snapshot/cmd.go
	new file:   client/snapshot/delete.go
	new file:   client/snapshot/dump.go
	new file:   client/snapshot/export.go
	new file:   client/snapshot/list.go
	new file:   client/snapshot/load.go
	new file:   client/snapshot/restore.go
	modified:   server/util.go
	modified:   snapshots/manager.go

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	both modified:   CHANGELOG.md
	both modified:   go.mod
	both modified:   server/types/app.go
	both modified:   simapp/simd/cmd/root.go
	deleted by us:   simapp/simd/cmd/root_v2.go
	both modified:   snapshots/store.go

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/github/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally


Mergify commands and options

More conditions and actions can be found in the documentation.

You can also trigger Mergify actions by commenting on this pull request:

  • @Mergifyio refresh will re-evaluate the rules
  • @Mergifyio rebase will rebase this PR on its base branch
  • @Mergifyio update will merge the base branch into this PR
  • @Mergifyio backport <destination> will backport this PR on <destination> branch

Additionally, on Mergify dashboard you can:

  • look at your merge queues
  • generate the Mergify configuration with the config editor.

Finally, you can contact us on https://mergify.com

Co-authored-by: Marko <marbar3778@yahoo.com>
(cherry picked from commit c1ceb3b)

# Conflicts:
#	CHANGELOG.md
#	go.mod
#	server/types/app.go
#	simapp/simd/cmd/root.go
#	simapp/simd/cmd/root_v2.go
#	snapshots/store.go
@mergify mergify bot requested a review from a team as a code owner May 11, 2023 10:15
@mergify mergify bot added the conflicts label May 11, 2023
}

path := args[0]
fp, err := os.Open(path)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
client/snapshot/dump.go Fixed Show fixed Hide fixed
return err
}

fp, err := os.Create(output)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
if err != nil {
return err
}
defer fp.Close()

Check failure

Code scanning / gosec

Deferring unsafe method "Close" on type "io.ReadCloser"

Deferring unsafe method "Close" on type "*os.File"
snapshots/store.go Fixed Show fixed Hide fixed
// The hash of the chunk is appended to the snapshot's metadata,
// and the overall snapshot hash is updated with the chunk content too.
func (s *Store) saveChunk(chunkBody io.ReadCloser, index uint32, snapshot *types.Snapshot, chunkHasher, snapshotHasher hash.Hash) error {
defer chunkBody.Close()

Check failure

Code scanning / gosec

Deferring unsafe method "Close" on type "io.ReadCloser"

Deferring unsafe method "Close" on type "io.ReadCloser"
if err != nil {
return errors.Wrapf(err, "failed to create snapshot chunk file %q", path)
}
defer chunkFile.Close()

Check failure

Code scanning / gosec

Deferring unsafe method "Close" on type "io.ReadCloser"

Deferring unsafe method "Close" on type "*os.File"
Comment on lines +68 to +77
go func() {
defer close(quitChan)

savedSnapshot, err := snapshotStore.Save(snapshot.Height, snapshot.Format, chunks)
if err != nil {
fmt.Println("failed to save snapshot", err)
return
}
quitChan <- savedSnapshot
}()

Check notice

Code scanning / CodeQL

Spawning a Go routine

Spawning a Go routine may be a possible source of non-determinism
"fmt"
"io"
"os"
"reflect"

Check notice

Code scanning / CodeQL

Sensitive package import

Certain system packages contain functions which may be a possible source of non-determinism

for i := uint32(0); i < snapshot.Chunks; i++ {
path := snapshotStore.PathChunk(height, uint32(format), i)
file, err := os.Open(path)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
if err != nil {
return fmt.Errorf("failed to open chunk file %s: %w", path, err)
}
defer file.Close()

Check failure

Code scanning / gosec

Deferring unsafe method "Close" on type "io.ReadCloser"

Deferring unsafe method "Close" on type "*os.File"
defer chunkBody.Close()

path := s.PathChunk(snapshot.Height, snapshot.Format, index)
chunkFile, err := os.Create(path)

Check failure

Code scanning / gosec

Potential file inclusion via variable

Potential file inclusion via variable
@julienrbrt
Copy link
Member

@yihuang can we improve the restore command (from main)? If we try loading a non-existing snapshot, it panics with a nil pointer.

@julienrbrt julienrbrt merged commit cbd8a01 into release/v0.47.x May 12, 2023
@julienrbrt julienrbrt deleted the mergify/bp/release/v0.47.x/pr-16067 branch May 12, 2023 10:38
@yihuang
Copy link
Collaborator

yihuang commented May 12, 2023

@yihuang can we improve the restore command (from main)? If we try loading a non-existing snapshot, it panics with a nil pointer.

sure

roy-dydx pushed a commit to dydxprotocol/cosmos-sdk that referenced this pull request Jul 11, 2023
…cosmos#16102)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
kakysha pushed a commit to InjectiveLabs/cosmos-sdk that referenced this pull request Aug 15, 2023
…cosmos#16102)

Co-authored-by: yihuang <huang@crypto.com>
Co-authored-by: marbar3778 <marbar3778@yahoo.com>
Co-authored-by: Julien Robert <julien@rbrt.fr>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

3 participants