Add support for HAMT directories

[Gozala]

JS-ipfs starts sharing directories once directory has more that 1000 entries, so that it fits in block limit.

We need to support sharded directories for the same reason.

[Gozala]

Blocked on getting coverage in Gozala/perma-map#1 to 100% so it can land and be released

[Gozala]

@alanshaw so have some code laying around but I don't think it's very useful. Most of basically does what I pulled out into separate repo Gozala/perma-map#1 idea was to rewrite the directory writer to use the hamt from the get go. Then on close asses whether it's small enough to be just encoded as flat directory (current behavior) or encode it as UnixFS HAMT. Also note that HAMT implementation is more generic and optimal that one used by UnixFS, however there is unixfs module in the repo that can map into UnixFS expected DAG layout and some tests verifying it produces same result as unixfs-importer.

[alanshaw]

I have a script to create a fixture from a CAR file with a HAMT sharded directory in it.

Idea is to generate a fixture do an ipfs add -r big-dir and then ipfs dag export bafy... > my.car and use the script to generate the fixture from the data.

I'm currently encountering a bug where some values are coming back incorrectly - they're the key not the value 😱:

[alanshaw]

Resolved by #41