-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use backwards-compatible formats during backup #3629
Conversation
This change converts the keys and posting lists to a backwards-compatible format so that backups work accross versions of Dgraph. The restore logic is also changed to convert the data back to the internal Dgraph formats.
fa7cd5f
to
20dcc59
Compare
This is now ready for review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 3 files at r1, 1 of 1 files at r3, 1 of 1 files at r4.
Reviewable status: all files reviewed, 11 unresolved discussions (waiting on @martinmr)
ee/backup/backup.go, line 108 at r4 (raw file):
stream := pr.DB.NewStreamAt(pr.Request.ReadTs) stream.LogPrefix = "Dgraph.Backup" stream.KeyToList = toBackupList(pr.Request.SinceTs)
pr.toBackupList
ee/backup/backup.go, line 183 at r4 (raw file):
func toBackupList(since uint64) func([]byte, *badger.Iterator) (*bpb.KVList, error) { return func(key []byte, itr *badger.Iterator) (*bpb.KVList, error) {
No need for this.
ee/backup/backup.go, line 186 at r4 (raw file):
list := &bpb.KVList{} loop:
No need for loop.
ee/backup/backup.go, line 190 at r4 (raw file):
item := itr.Item() if !bytes.Equal(item.Key(), key) { break
return?
ee/backup/backup.go, line 195 at r4 (raw file):
// Ignore versions less than given timestamp, or skip older versions of // the given key. break
return
ee/backup/backup.go, line 250 at r4 (raw file):
if item.DiscardEarlierVersions() || item.IsDeletedOrExpired() { break loop
return
ee/backup/restore.go, line 42 at r4 (raw file):
dir := filepath.Join(pdir, fmt.Sprintf("p%d", groupId)) db, err := badger.OpenManaged(badger.DefaultOptions(dir). WithSyncWrites(true).
Set it to false.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: all files reviewed, 11 unresolved discussions (waiting on @golangcibot and @manishrjain)
ee/backup/backup.go, line 180 at r1 (raw file):
Previously, golangcibot (Bot from GolangCI) wrote…
unreachable: unreachable code (from
govet
)
Done.
ee/backup/backup.go, line 200 at r2 (raw file):
Previously, golangcibot (Bot from GolangCI) wrote…
ineffectual assignment to
err
(fromineffassign
)
Done.
ee/backup/backup.go, line 108 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
pr.toBackupList
Done.
ee/backup/backup.go, line 183 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
No need for this.
Done.
ee/backup/backup.go, line 186 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
No need for loop.
Done.
ee/backup/backup.go, line 190 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
return?
Done.
ee/backup/backup.go, line 195 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
return
Done.
ee/backup/backup.go, line 250 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
return
Done.
ee/backup/restore.go, line 26 at r1 (raw file):
Previously, golangcibot (Bot from GolangCI) wrote…
File is not
gofmt
-ed with-s
(fromgofmt
)
Done.
ee/backup/restore.go, line 36 at r1 (raw file):
Previously, golangcibot (Bot from GolangCI) wrote…
func parameter
backupId
should bebackupID
(fromgolint
)
Ignoring
ee/backup/restore.go, line 42 at r4 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Set it to false.
Done.
This change converts the keys and posting lists to a backwards-compatible format so that backups work accross versions of Dgraph. The restore logic is also changed to convert the data back to the internal Dgraph formats.
This change converts the keys and posting lists to a
backwards-compatible format so that backups work accross versions of
Dgraph. The restore logic is also changed to convert the data back to
the internal Dgraph formats.
This change is