fix: interoperability with go repo

[vasco-santos]

When a go-ipfs daemon runs the init command, it creates a datastore_spec file and the config file has the datastore property on it. This way, when a go daemon stars, it expects to have those two in its repo.

Aiming to have interoperability between go and ipfs repo (interop#26 and interop#28), the js-ipfs repo has to have both in its repo.

• Add datastore default config to js-ipfs js-ipfs#1460 (instead of directly here)
• Add running test in ipfs/interop for the repo interoperability interop#29

Fixes #163

[jacobheun]

If i customize the storageBackends when creating the repo, https://github.com/ipfs/js-ipfs-repo#new-repopath-options, it should probably fail if the datastore.Spec file does not match.

This PR is supporting the default and gives us the ability to customize, but it also lets the actual repo behavior differ from the spec file. If go runs on a repo that js has a custom config for and the spec file doesn't reflect that, bad things will probably happen.

[vasco-santos]

Yeah @jacobheun ! That is one of the problems that I would like to discuss. The ability to customize may be problematic because go-ipfs has different types of datastores, which are still not available on js-ipfs.

[jacobheun]

That's the downside of customizing, you start losing compatibility. Out of the box we want js and go to interop without issue, but as people change that default configuration we can't guarantee that's the case anymore. I think it's something we can put in the readme when we update it for this to say, "If you change this, you risk losing ipfs-repo interoperability with other languages (go, etc)".

If we don't recognize the spec or we can't find the matching datastores, sharding methods, etc, it should just throw an incompatibility error and exit. No touching the data if it doesn't know how to handle it properly.

[vasco-santos]

After a productive call with @jacobheun we created an endeavour issue to tackle all the tasks regarding the repo interoperability. This issue aims to tackle the following set of tasks:

3) ipfs/js-ipfs-repo

• 1. Throw an error on invalid / inconsistent configs
  • INIT: create the repo based on the config received (verify if the storageBackends exists and if so, guarantee that it matches the spec). If is is not possible to create the repo an error should be thrown to inform that the options received are not valid.
  • Start: verify if the repo that exists is compliant with the configuration. If not, throw an error
• 2. transition for using spec instead of the current programatic approach (support both to avoid breaking existing projects, but deprecate the current format)

[jacobheun]

I tested these agains the interop https://github.com/ipfs/interop/pull/29/files unskip updates and they look good. This doesn't include 2.2, 3.1 and 3.2 from the awesome endeavor. We will need to include those in the next piece of the update, but this gets us our initial interop functionality.