Build folder structures given path with names.

[travisperson]

This will be possible given #3 is incorporated. Folder sturctures can be send to IFPS by nesting multipart/form-data structures into a single part.

Example request:

User-Agent: /node-ipfs-webui/0.1.0
Content-Type: multipart/form-data; boundary=knyfc6alrmu9dx6rfis020litgknvcxr

--knyfc6alrmu9dx6rfis020litgknvcxr
Content-Type: multipart/form-data; boundary=lugqob2u0zjb57b9nz5l6ggohxrjxlxr
Content-Disposition: folder; name="folder"; filename="afolder"

--lugqob2u0zjb57b9nz5l6ggohxrjxlxr
Content-Type: application/octet-stream
Content-Disposition: file; name="file"; filename="FileA.txt"

A file inside of a folder!
--lugqob2u0zjb57b9nz5l6ggohxrjxlxr
Content-Type: application/octet-stream
Content-Disposition: file; name="file"; filename="FileB.txt"

Another file inside of a folder!
--lugqob2u0zjb57b9nz5l6ggohxrjxlxr--
--knyfc6alrmu9dx6rfis020litgknvcxr--

[jbenet]

👍

[travisperson]

Quick hack at this using vinyl files.

https://gist.github.com/travisperson/f68c8e68ae325d742a29

[krl]

The question, as i see it, is: where do we construct the unixfs objects.

One way is piping directory info into ipfs, another is constructing unixfs objects client-side, and just referencing them.

Client side unixfs-modeling is something i've begun to sketch out in my head, to build ontop https://github.com/krl/node-ipfs-mdag

This could also go towards making client thicker, and eventually drop the backend dep.

We could do this, and work on the other model at the same time though, as this looks like great code to have right now.

[travisperson]

I think the node-ipfs-mdag is a good module (and very much needed for node-ipfs), but not for node-ipfs-api. I think the primary goal of this module is to communicate to an ipfs node (be it a NodeJS/iojs, go or rust implementation).

What I envision this module as is just a communication layer that acts just like the cli.

[travisperson]

Made a few changes to this, but primarily it's about the structure of the vinyl files.

I wanted an easy way to allow for wrapping of directories. What I've come up with is the following inputs to outputs.

Wrapped files in a directory

files.push(new File({
    cwd:  '/',
    base: '/base/',
    path: '/base/dir/hello.txt',
    contents: new Buffer("world")
}))

files.push(new File({
    cwd:  '/',
    base: '/base/',
    path: '/base/dir/ipfs.txt',
    contents: new Buffer("is cool")
}))

files.push(new File({
    cwd:  '/',
    base: '/base/',
    path: '/base/readme.md',
    contents: new Buffer("wow")
}))

files.push(new File({
    cwd:  '/',
    base: '/base/',
    path: '/base/dir/other/readme.md',
    contents: new Buffer("wow")
}))

{
  "Name": "base/readme.md",
  "Hash": "QmbxMp5x7RbqHSTrrKQCYjccw2obvgcz2GEyz3gEkixhDS"
}
{
  "Name": "base/dir/hello.txt",
  "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk"
}
{
  "Name": "base/dir/ipfs.txt",
  "Hash": "QmfBfWxn32sGXbkgDoeNLT5rBcTehYffMyeU5LFTTByt6V"
}
{
  "Name": "base/dir/other/readme.md",
  "Hash": "QmbxMp5x7RbqHSTrrKQCYjccw2obvgcz2GEyz3gEkixhDS"
}
{
  "Name": "base/dir/other",
  "Hash": "QmSV3qfAxjMTdGrB5tGA4daPsL6NoZKxhUx6QxCfqLCeGp"
}
{
  "Name": "base/dir",
  "Hash": "QmeE8ZjMwexLbZBAX1dxwguCaQ5xyHw4wYGmGiyfzEqL2p"
}
{
  "Name": "base",
  "Hash": "QmeUg1UYEAmrZZWW7URFZXMYMVECzfmgdgke95F8HqauLz"
}

Grouping of files

files.push(new File({
    cwd:  '/base/',
    base: '/base/',
    path: '/base/dir/hello.txt',
    contents: new Buffer("world")
}))

files.push(new File({
    cwd:  '/base/',
    base: '/base/',
    path: '/base/dir/ipfs.txt',
    contents: new Buffer("is cool")
}))

files.push(new File({
    cwd:  '/base/',
    base: '/base/',
    path: '/base/readme.md',
    contents: new Buffer("wow")
}))

files.push(new File({
    cwd:  '/base/',
    base: '/base/',
    path: '/base/dir/other/readme.md',
    contents: new Buffer("wow")
}))

{
  "Name": "readme.md",
  "Hash": "QmbxMp5x7RbqHSTrrKQCYjccw2obvgcz2GEyz3gEkixhDS"
}
{
  "Name": "dir/hello.txt",
  "Hash": "QmNyJpQkU1cEkBwMDhDNFstr42q55mqG5GE5Mgwug4xyGk"
}
{
  "Name": "dir/ipfs.txt",
  "Hash": "QmfBfWxn32sGXbkgDoeNLT5rBcTehYffMyeU5LFTTByt6V"
}
{
  "Name": "dir/other/readme.md",
  "Hash": "QmbxMp5x7RbqHSTrrKQCYjccw2obvgcz2GEyz3gEkixhDS"
}
{
  "Name": "dir/other",
  "Hash": "QmSV3qfAxjMTdGrB5tGA4daPsL6NoZKxhUx6QxCfqLCeGp"
}
{
  "Name": "dir",
  "Hash": "QmeE8ZjMwexLbZBAX1dxwguCaQ5xyHw4wYGmGiyfzEqL2p"
}

[travisperson]

Just has a sudden realization that I should support vinyl-fs if I am going to use vinyl files. Testing to see if this all still works.

[jbenet]

@krl

where do we construct the unixfs objects.

One way is piping directory info into ipfs, another is constructing unixfs objects client-side, and just referencing them.

either way should be possible with a full suite of node modules, and work fine. in the meantime, whatever you prefer (piping to ipfs is fine from ipfs-api as @travisperson points out, but not so fine if we cant assume a daemon exists (like if we wrote other tools that use merkledag + unixfs formats, like a ipfs-unixfs-stream or something that does not assume ipfs exists))

---

@travisperson the "Name": "dir/other/readme.md", is scary. fine to carry it around with vinyl stuff, but carful to make sure stuff isn't piped straight into ipfs-- slashes aren't put into ipfs link names. i had this as a "flattened tree" idea in DRAFT 3 paper, but am not so sure it's a good idea anymore-- it gets hard to resolve things, precedence of links gets hard. Example:

A: 
  foo:  <hash-of-B>
  foo/bar: <hash-of-C>

B: 
  bar: <hash-of-D>

C: 
  data: "hey, i'm c"

D: 
  data: "hey, i'm d"

> ipfs object data /ipfs/<hash-of-A>/foo/bar
# should this be C or D?

[travisperson]

I did it with the full path to mimic the ipfs cli.

~/s/g/i/node-ipfs-api (feat/streams) $ ipfs add -r base/
added QmaRGe7bVmVaLmxbrMiVNXqW4pRNNp3xq7hFtyRKA3mtJL base/dir/hello.txt
added QmbEHC62SxkfM1hjKeUVi5wb1LrrSBmuRKWspXRz4vr3X5 base/dir/ipfs.txt
added QmTppEckQeay4RZ5meNCSsUYbNDwVdYtd1FNz5vqRb3pap base/dir/other/readme.md
added QmTq4bTeDKxGrUBF7tokf2zys5WJRfwdjNyiWRt6NDdvXw base/dir/other
added QmTMmxos8sCqvpJxkc4xZerPBjpVKU49grf9yy46REzMpD base/dir
added QmTppEckQeay4RZ5meNCSsUYbNDwVdYtd1FNz5vqRb3pap base/readme.md

The paths themselves don't go into the object, the daemon takes care of that for us.

~/s/g/i/node-ipfs-api (feat/streams) $ ipfs object get QmYSQFAGNhASRMPdjzBYTqpRM2fVpWMLyptMTGxdMn8sjE/dir/other
{
  "Links": [
    {
      "Name": "readme.md",
      "Hash": "QmTppEckQeay4RZ5meNCSsUYbNDwVdYtd1FNz5vqRb3pap",
      "Size": 12
    }
  ],
  "Data": "\u0008\u0001"
}⏎ 

[jbenet]

@travisperson yay :) just making sure.