IPFS Repo Spec update

[jbenet]

The repo spec has some discrepancies to fix. I've added some commits that fix some, and noted others. This PR is a discussion to fix it.

Thanks to @masylum for prompting this fix, and sorry you had to find it by hand.

Discrepancies to address:

• fs-repo: version format
• fs-repo: pid in repo.lock
• fs-repo: ipfs repo fsck to fix lock issues.
• fs-repo: blocks dir layout (fanout/depth)
• fs-repo: come up with a name for api-file-only repo

cc @ion1 @whyrusleeping @diasdavid

[masylum]

I think it would be good to specify the schema. Ideally any implementation should be compatible. Also, mention that its currently stored in JSON.

[jbenet]

that sounds good-- though i do want to keep config files flexible, in that people may want to add more values into it than we give -- similar to how people use .git/config

[masylum]

Good to see specs moving forward little by little 💪

My two cents: I see some overlap between fs-repo and the repo specs. For instance, I'm not sure repo.lock is something a non-fs repo should be worried about. I'm thinking, for instance of an IndexedDB/Localstorage implementation for the browser, it probably does not care about locks at that level.This week I will try to move forward with the js implementation. Those differences will become more obvious when trying to get a non fs based repository.

[jbenet]

I'm not sure repo.lock is something a non-fs repo should be worried about. I'm thinking, for instance of an IndexedDB/Localstorage implementation for the browser, it probably does not care about locks at that level.

makes sense. I only included that in repo spec to make it clear people need to deal with concurrent access. we could just make it a clear warning and remove the repo.lock requirement there altogether.

any other observations like that?

Those differences will become more obvious when trying to get a non fs based repository.

sounds great!

thanks for the feedback-- a solid spec here will go a long way

[daviddias]

It would be pretty sweet to upgrade https://github.com/maxogden/abstract-blob-store to have a parallel interface for Go, so that we can have all of those s3/fs/mem repos for 'free'.

[jbenet]

In go, we use https://github.com/jbenet/go-datastore/ -- which is similar and alraedy has a bunch.

i've been making these abstractions for a long time: https://github.com/datastore/datastore and https://github.com/datastore