Provide migration tool from v1 to v2 and viceversa

[campoy]

As we are going to release the first data backward-incompatible changes we should also provide an easy way to transform a dataset from v1 to v2 and back.

This should be an easy task and it doesn't need to even be part of badger, it could be badger-migrate repository that imports badger at v1 and v2 and does the necessary transformations.

[martinmr]

I checked the code and the backups are written in the KV proto format so as long as that doesn't change, backups should work between versions. That proto has not changed since December 2018. I think we can close this for now.

[campoy]

Thanks for the update, @martinmr.

In that case I don't think this tool is that useful anymore, as one could migrate from one version to the other by following these steps:

1. start with a dataset in vX and with the badger binary installed for the same version
2. run badger backup --dir bak
3. download the badger binary for vY
4. run badger restore --dir -bak
5. this will generate a new directory containing all the initial data in the new version

[manishrjain]

Yes, doing a backup and then restoring should work. But, it would be slow and a lot more resource intensive. Instead, we can use the stream framework to read data from 1 and write it back using stream writer at the speed of disk, i.e. >200MBps -- that'd be much faster. Simple script like this can save many hours cumulatively for our users.

[ashish-goswami]

I also made a change in commit 4e5cbcc, which changes backup file format(marshal entry list instead of single entry). Hence backups taken before this commit cannot be restored using current code.

[connorgorman]

I think this issue is still relevant as @ashish-goswami has mentioned. However, I think a simple solution could be to write some magic and version at the beginning of the file (kind of like the manifest magic) and then branch off to choose the proper backup mechanism