IPFS extensibility

[JGAntunes]

I was wondering if are there any plans or even if it was ever raised to discussion the ability to extend IPFS API with custom commands/operations?

For reference, what motivated me to consider this was my work with #266. For now I've hacked together an IPFS fork that has PulsarCast as a drop in replacement for the floodsub module. However, these have different APIs and features that don't exactly feat well under the same interface. While wondering on what a long term solution for this could be I came to realise that that's probably fine, these should probably be different commands/operations under an IPFS node. At the same time, I looked into something like what Kubernetes provides (the ability to define custom resources and controllers) and it stroke me how nice would it be to have something like this in IPFS (and how well it would fit my use case/problem). I do realise these are totally different pieces of software, but I do see a lot of potential on the ability to define a custom command/operation (exposing it in IPFS HTTP API, cli and such) with a custom controller/handler that could have access to IPFS core and interact with IPLD and libp2p. This would even fit really well with modularization approach followed by the project :) in the limit all the commands/operations in IPFS core could even be plugable 🙌

[Stebalien]

So, there are several things we'd like to do in this area:

1. Create a separate libp2p daemon. Eventually, I'd like go-ipfs to use this daemon as a shared libp2p "swarm". That way, pubsub modules etc. would just be additional applications (using a shared DHT, libp2p swarm, etc.).
2. Create a separate data daemon for storing and exchanging data.
3. Improve our plugin system. Ideally, users would be able to add alternative pubsub routing algorithms, data exchange protocols, DHTs, etc. However, as you noted, the tricky part is getting the interfaces right.

For now, the "closest" solutions are:

1. Creating an entirely new application directly on top of libp2p, IPLD, and bitswap (especially if you don't need the rest of go-ipfs).
2. Improving the plugin system.
3. Pushing forward work on making go-ipfs usable as a library (CoreAPI progress tracking issue kubo#4498).