IPLD with Network & Persistence

[Gozala]

Currently we have:

1. libp2p - networking layer
2. IPLD - a data layer
3. IPFS - a file system and a network (with all the above and more)

For years I have being building things with IPFS even though, what I really used was 1. and 2. with the network & persistance of the IPFS. Recently I started applying some of my learning to IPFS in browsers (ipfs/js-ipfs#3022) and talking to the various teams in our community to inform this work. Through this conversations I have noticed that:

• All prominent users of IPFS (in browser context), that I had a chance to talk, to use:
  • Primarily ipfs.dag API
  • Bits of libp2p (like pubsub) for replication.
• js-ipfs-lite from Textile basically is IPLD with Network & Persistence

Then when I think about it makes sense that people use js-ipfs (when they really want DAG with network & persistence) because that is the only thing that puts all the pieces together. Which is to suggest that if we had a thing that was just IPLD with persistence and network (of the IPFS) that is what all these teams would use. In fact that is more or less what shared IPFS node ipfs/js-ipfs#3022 ended up.

I think this it is worth considering to break out another layer (just like libp2p and ipld come to be) from the IPFS that is IPLD + Network + Persistence (let's call it a DAGService for now). I think the evidence of the demand is pretty clear, but besides I see few compelling reasons to do so:

1. It creates an opportunity to reduce the scope of IPFS and focus on FS piece.
2. Enables us to iterate on the API that fits this demand.
3. Creates a light reusable component that actually addresses user needs.

[carsonfarmer]

I've weighed my support for this in other contexts as well, but just to reiterate here. This is a fantastic idea. Obviously I'm slightly biased in that I am the primary author of js-ipfs-lite, but that work was based on actual, real-world, practical usage of IPFS in Javascript. It ticked a bunch of boxes for our team, and has proven to be relatively popular with other teams as well. Unfortunately, it has been difficult to prioritize maintenance of that package with our other commitments, so to see if breath new life would be fantastic. I think it provides a pretty solid basis for exactly what @Gozala is outlining here. And I can all but guarantee our team would be heavy users.

[jimpick]

Peer-base (used by Peerpad) basically just uses js-ipfs for libp2p + the raw datastore for persistance (no IPLD/IPFS). It would make sense to use lighter dependencies if they were available so there's less to load over-the-wire.