Use manifest to only restore preds assigned to each group. #3648
Conversation
This change converts the keys and posting lists to a backwards-compatible format so that backups work accross versions of Dgraph. The restore logic is also changed to convert the data back to the internal Dgraph formats.
This PR changes the restore logic so that only the keys corresponding to the predicates that were assigned to the group at the time of the last backup are restored. This change is intended to properly deal with predicate moves.
|
Marked as draft since the base is another branch currently on review but this change is ready for review. |
There was a problem hiding this comment.
Reviewed 5 of 5 files at r1.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @martinmr)
ee/backup/restore.go, line 58 at r1 (raw file):
return nil } return loadFromBackup(db, gzReader, 16, preds)
Remove 16 from here.
ee/backup/restore.go, line 64 at r1 (raw file):
// loadFromBackup reads the backup, converts the keys and values to the required format, // and loads them to the given badger DB. func loadFromBackup(db *badger.DB, r io.Reader, maxPendingWrites int, preds predicateSet) error {
Remove maxPendingWrites here.
There was a problem hiding this comment.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @martinmr)
There was a problem hiding this comment.
Reviewable status: 3 of 5 files reviewed, 2 unresolved discussions (waiting on @manishrjain)
ee/backup/restore.go, line 58 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Remove 16 from here.
Done.
ee/backup/restore.go, line 64 at r1 (raw file):
Previously, manishrjain (Manish R Jain) wrote…
Remove maxPendingWrites here.
Done.
…#3648) This PR changes the restore logic so that only the keys corresponding to the predicates that were assigned to the group at the time of the last backup are restored. This change is intended to properly deal with predicate moves.
This PR changes the restore logic so that only the keys corresponding to
the predicates that were assigned to the group at the time of the last
backup are restored. This change is intended to properly deal with
predicate moves.
This change is