Rename ipfs files commands to ipfs mfs

[schomatis]

I'm not even sure we can do this without breaking too much stuff but I wanted to draw some attention to the fact that our family of ipfs files command, documented as

  ipfs files - Interact with unixfs files.

  Files is an API for manipulating IPFS objects as if they were a unix
  filesystem.

actually seems to deal exclusively with MFS (not just any UnixFS file), that is (rewording this question), if I add content with ipfs add creating a UnixFS file, that file actually won't be accessible with the ipfs files command (e.g., ipfs files ls) since I need to add it to the MFS first. There doesn't seem to be any clear hint about this, not in the name of the command nor in its documentation.

[Stebalien]

I agree this is really confusing.

I believe the (nebulous) plan is to go the other way and:

1. Make ipfs add SomeFile add that file to MFS.
2. Make ipfs {add,cp,...} an alias for ipfs files *.
3. Make ipfs files CMD /ipfs/... "just work".

However, we've been holding off on features like this until we're ready for an API v2. I'm not sure if there are even any design docs for this yet.

[Stebalien]

Adding an ipfs files add command may help.

[nitishm]

if I add content with ipfs add creating a UnixFS file, that file actually won't be accessible with the ipfs files command (e.g., ipfs files ls) since I need to add it to the MFS first. There doesn't seem to be any clear hint about this, not in the name of the command nor in its documentation.

What I dont understand is in ipfs add we create an MFS root object in the Adder.addNode() method (albeit with unixfs.EmptyDirNode()), why is it that ipfs files ls doesn't find it ?

[nitishm]

Might be my lack of exposure to IPFS but another thing that confuses me is ipfs file vs ipfs files *. Why does the file command exist, if all it provides is an ls subcommand. Could both these commands be collapsed into one ?

[schomatis]

What I dont understand is in ipfs add we create an MFS root object in the Adder.addNode() method (albeit with unixfs.EmptyDirNode()), why is it that ipfs files ls doesn't find it ?

Yes, another great question (please add it to the documentation issue).

The short answer is: we "discard" it, in the sense that we return its hash to the user if the --wrap-with-directory option is set or we just don't even do that if not, we'll pin its contents (by default), you'll be able to access it (with something like ipfs cat) but that is not the MFS root we access with ipfs files, let's call that one our local MFS root (single, the same one we'll access over and over again across different ipfs files calls). Why do we use these ephemeral MFS roots in ipfs add in the first place? I think because of the benefits of working in the MFS abstraction layer.

(When I say great question I mean it, because I didn't write the answer off the top of my head, I had to go back to the code and check, and I had to think of the difference between the ipfs add MFS root and the one in ipfs files, and I had to start making up new terms like local vs ephemeral MFS roots to accommodate this information to the MFS mental model in my head, so, thanks!)

[schomatis]

Could both these commands be collapsed into one ?

Yes, we've discussed this many times across different issues but I'm not sure what was the final decision..

[nitishm]

but that is not the MFS root we access with ipfs files, let's call that one our local MFS root

I have got to read through the the files code to understand this better !

[schomatis]

I have got to read through the the files code to understand this better !

Actually you'll need to check

https://github.com/ipfs/go-ipfs/blob/c17aaa6e26512b6bbd0f60c0c39c82daacfba9e1/core/core.go#L858-L863

https://github.com/ipfs/go-ipfs/blob/c17aaa6e26512b6bbd0f60c0c39c82daacfba9e1/core/core.go#L895-L901

(although you may need to get a sense of our republisher system to understand this, basically when we flush a file or close the MFS root -daemon shutdown or end of a standalone CLI command- the publish pf method is called).

[ec1oud]

So you mean IPFS will actually look like a hierarchical filesystem some day? It's been years already without having that feature, so it looks to me like mfs/unixfs is a subset of the whole IPFS datastore: you can use it as a local filesystem, but it's mainly not for sharing; whereas when you do "ipfs add" you just get a hash, which you can share, and that's intended more for publishing web sites and such. But it sortof makes sense to have everything accessible in a single-rooted hierarchy like a filesystem, but is that really where you guys are going? It's a much more conventional way of working. I'm trying to build a graphical client (combined editor/browser) so I'm baking in my assumptions about how things work, and then suddenly I read this bug and find that maybe I don't understand where it's going at all.

Also, files/ls doesn't provide the hashes of the files by default (but that's mentioned in #5026) which seems like an odd choice if the idea is to first add files to unixfs and then share the hashes of some subdirs, for example. But if the intention is to combine ipfs files ls and ipfs ls into one command (#5806) I hope that gets solved too.

[Stebalien]

But it sortof makes sense to have everything accessible in a single-rooted hierarchy like a filesystem, but is that really where you guys are going? It's a much more conventional way of working.

Personally, I'd like to move in this direction. However, we probably won't for a while.

However, form a UX perspective, I recommend exposing MFS to users rather than pins. That's what the ipfs-desktop and the current WebUI do, as far as I know.

Also, files/ls doesn't provide the hashes of the files by default (but that's mentioned in #5026) which seems like an odd choice if the idea is to first add files to unixfs and then share the hashes of some subdirs, for example.

A concern here is performance. Ideally, we wouldn't be forced to materialize hashes of anything until we need them. To get the hash, you'd call ipfs files stat /path/to/thing.