Add support for --only-hash - Calculating a multihash without adding the content to IPFS

[fazo96]

• Version: 0.27.7
• Platform: any

Type:

Enhancement

Severity:

Low

Description:

go-ipfs has the --only-hash option when calling ipfs add. It is useful in the strange case where you need to store a multihash of some data somewhere before actually adding the data to IPFS.

Maybe this is already possibile? I tried looking around but it seems the option is only in go-ipfs. I thought about adding the data then deleting from the local node but it looks like this can't be done either

[mitra42]

While I don't need this currently, this was something one of our earlier demos wanted. The scenario was building a data structure that referred to a IPFS, I wanted something that would synchronously return the hash to add to that data structure while I was in the background asynchronously adding the content.

[fazo96]

We found that using https://github.com/ipld/js-ipld-dag-pb (for protobuf) and https://github.com/ipld/js-ipld-dag-cbor (for cbor) directly allows us to easily compute hashes, however this cannot be used when adding a directory tree, only with small self-contained pieces of information

[JonKrone]

@fazo96, there was some discussion of --only-hash in ipfs-inactive/js-ipfs-http-client#509. Is that what you're looking for? I looked at adding it and it's on my list but haven't found the time yet to work on it. You're welcome to take it on if you're interested :)

[fazo96]

@JonKrone yes, that's exactly what I was looking for, I think it should also be implemented in js-ipfs directly not only in the api client

For now my problem is solved by creating DAGNodes directly, I'd still like to contribute but I need to see if I can find the time

[alexanderattar]

@fazo96 can you elaborate a little using DAGNode.create to compute the hash? I so far have been unable to get hashes to match from node.files.add. Here's an example that demonstrated different CIDs for each output:

'use strict'

const series = require('async/series')
const IPFS = require('ipfs')
const {
  DAGNode,
  DAGLink
} = require('ipld-dag-pb')

const node = new IPFS()
let fileMultihash

series([
  (cb) => node.on('ready', cb),
  (cb) => node.version((err, version) => {
    if (err) { return cb(err) }
    console.log('Version:', version.version)
    cb()
  }),
  (cb) => DAGNode.create(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash:', mh)
    cb(null, mh)
  }),
  (cb) => node.files.add(Buffer.from('Hello'), (err, filesAdded) => {
    if (err) { return cb(err) }
    console.log('\nAdded file:', filesAdded[0].path, filesAdded[0].hash)
    fileMultihash = filesAdded[0].hash
    cb()
  }),
  (cb) => node.files.cat(fileMultihash, (err, data) => {
    if (err) { return cb(err) }
    console.log('\nFile content:')
    process.stdout.write(data)
  })
])

[fazo96]

@alexanderattar In my case what I needed was the same result between creating a DAGNode (without storing it in IPFS) using DAGNode.create(buffer) and the result of calling ipfs.object.put(buffer) where the buffer is identical between the two.

This works, I modified your example to show it:

'use strict'

const series = require('async/series')
const IPFS = require('ipfs')
const {
  DAGNode,
  DAGLink
} = require('ipld-dag-pb')

const node = new IPFS()
let fileMultihash

series([
  (cb) => node.on('ready', cb),
  (cb) => node.version((err, version) => {
    if (err) { return cb(err) }
    console.log('Version:', version.version)
    cb()
  }),
  (cb) => DAGNode.create(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash (dagnode):', mh)
    cb(null, mh)
  }),
  (cb) => node.object.put(Buffer.from('Hello'), (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    const mh = dagNode.toJSON().multihash
    console.log('\nComputed hash (put):', mh)
    console.log('\nBuffer (put):', dagNode.toJSON().data)
    cb(null, mh)
  }),
  (cb) => node.files.add(Buffer.from('Hello'), (err, filesAdded) => {
    if (err) { return cb(err) }
    console.log('\nAdded file:', filesAdded[0].path, filesAdded[0].hash)
    fileMultihash = filesAdded[0].hash
    cb()
  }),
  (cb) => node.object.get(fileMultihash, (err, dagNode) => {
    if (err) {
      return cb(err)
    }
    console.log('\nBuffer (get after add):', dagNode.toJSON().data)
    cb(null, dagNode.toJSON().multihash)
  }),
  (cb) => node.files.cat(fileMultihash, (err, data) => {
    if (err) { return cb(err) }
    console.log('\nFile content:')
    process.stdout.write(data)
  })
])

I also tried to diff the data of the content added using files.add and added using object.put in the same script. It looks like there are some bytes added before the actual content when files.add is used which would explain the different hash.

I am not sure why this happens though, probably the files API is higher level and does some extra work

[alexanderattar]

Awesome! Thank you @fazo96. This is helpful