support threshold based automatic sharding and unsharding of directories

[schomatis]

Update UnixFS dependency which now handles HAMT transition internally:

• Remove the deprecated UseHAMTSharding option.
• Remove the addUnixFSChild intermediary function that was created only to handle HAMT transition. Instead we now add a child interacting with the UnixFS structure directly.

Closes #87.
Closes #86.

[hsanjuan]

Hey, a breaking go-unixfs release was made but it fails when mixed with the current go-mfs release because this is not merged, so it would be good to merge and tag.

[hsanjuan]

Should be bumping go-unixfs version in go.mod?

[b5]

would love to see this addressed! Seeing undefined: "github.com/ipfs/go-unixfs/io".UseHAMTSharding on any build that touches MFS.

[aschmahmann]

@b5 you should be able to just use the version of go-unixfs used by go-ipfs (v0.2.5) and things should be fine.

Note: go-unixfs v0.2.5 should not have been released, pre v2 go modules are as an ecosystem a bit confusing since things really need to upgrade together which means a release in a dependency shouldn't really happen until most of the important higher ups are ready to be released. When you don't you get issues like the one you're running into.

Meanwhile this is likely blocked on the automatic unixfs unsharding work completing ipfs/kubo#8106 which is underway.

[b5]

@b5 you should be able to just use the version of go-unixfs used by go-ipfs (v0.2.5) and things should be fine.

Sadly that's my problem. I'm filing PRs into projects that use go-unixfs@v0.2.6, with a codebase that has go-ipfs@v0.9.1+ in it's dependencies.

Totally agreed this is just life in go.mod land. I'm just calling attention to this specific conflict 😄, and can work around it if needed

[iand]

Can this be updated and merged? (I volunteer to fix conflicts). As it stands go-mfs is unusable with any modern arrangement of ipfs libs. I don't see why fixing that should be blocked behind automatic unsharding.