11-to-12: Re-work logic to use the backup file for both migrate and revert #150
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…r of workers and the sync size
Avoid that there is confusion about the unit used.
…evert Before: we Query all keys and write a backup file. Then Query all keys and perform the swaps. This may cause that our query keeps getting results for all the keys that we are writing while we run it. After: we perform a Prepare() step where we query all the keys and write the backup file. Then we read the backup file to perform the swaps without making another query. This also means that the migration (swapper.Run()) becomes very similar to the revert (swapper.Revert()), since both read from the backup file. We are thus able to refactor code and re-use logic better. We have also tuned down the noise when encountering multihashes that are not CIDv0s.
|
@aschmahmann this is a follow up to #149 . It addresses issues we discussed:
|
|
2022-01-25 discussion: @aschmahmann is going to review and also add some more tests. |
aschmahmann
approved these changes
Feb 15, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Before: we Query all keys and write a backup file. Then Query all keys and
perform the swaps. This may cause that our query keeps getting results for all
the keys that we are writing while we run it.
After: we perform a Prepare() step where we query all the keys and write the
backup file. Then we read the backup file to perform the swaps without making
another query.
This also means that the migration (swapper.Run()) becomes very similar to the
revert (swapper.Revert()), since both read from the backup file. We are thus
able to refactor code and re-use logic better.
We have also tuned down the noise when encountering multihashes that are not
CIDv0s.