ipfs.files.add(Buffer) > TypeError: content.once is not a function

[joaosantos15]

Hello, I'm using the ipfs-js-api and am having a problem with ipfs.files.add() when adding a Buffer.

I'm following the this usage example from js-ipfs-api/examples/name-api/index.js:

let buffer = Buffer.from(input.value)

  ipfs.add(buffer)
    .then(res => {
      publish(res[0].path)

      input.value = ''
    })
    .catch(err => {
      showStatus('failed to add the data', COLORS.error)
      console.error(err)
    })
}

This is my code:

var Promise = require('promise')
var Buffer = require('buffer/').Buffer
var ipfsAPI = require('ipfs-api')
var ipfs = ipfsAPI('/ip4/127.0.0.1/tcp/5001')

var myBuffer = Buffer.from('hello')

ipfs.files.add(myBuffer, function (err, result) {
  if (err) {
    console.log(err)
  } else {
    console.log(result[0].hash)
  }
})

And I'm getting this output (in Node.js):

Error: "files" must be a buffer, readable stream, or array of objects

In the browser, I can solve the problem by putting the Buffer inside an array (I don't know why that solves it though...):

ipfs.files.add([myBuffer], function (err, result) {

But in Node.js I get this error:

TypeError: content.once is not a function
    at Multipart._pushFile ($proj/src/node_modules/ipfs-api/src/utils/multipart.js:80:13)
    at Multipart._maybeDrain ($proj/src/node_modules/ipfs-api/src/utils/multipart.js:48:14)
    at Multipart._transform ($proj/src/node_modules/ipfs-api/src/utils/multipart.js:40:10)
    at Multipart.Transform._read (_stream_transform.js:167:10)
    at Multipart.Transform._write (_stream_transform.js:155:12)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Multipart.Writable.write (_stream_writable.js:220:11)
    at eachSeries ($proj/src/node_modules/ipfs-api/src/utils/send-files-stream.js:52:35)
    at $proj/src/node_modules/async/internal/withoutIndex.js:9:16

Does anyone have any idea why this happens?
Thanks :)

[joaosantos15]

I figured it out. 😅
Turns out I, in Node.js, was using a Buffer package that is not supported. That's why in Node I got the
Error: "files" must be a buffer, readable stream, or array of objects, as it was failing the check:

 const ok = Buffer.isBuffer(_files) ||
               isStream.readable(_files) ||
               Array.isArray(_files)

[daviddias]

@joaosantos15 actually, it should work as Buffer should just be a shim to replicate all the same functionality. This is a bug, we should test with the Buffer module on that assertion.

[joaosantos15]

@diasdavid Got it.
The error is being thrown in this verification:

 const ok = Buffer.isBuffer(_files) ||
               isStream.readable(_files) ||
               Array.isArray(_files)

I tried this in Node,

var Buffer2 = require('buffer/').Buffer

var newBuffer = Buffer2.from('hello')

console.log(Buffer.isBuffer(newBuffer))

And got false. Do you think that by changing the const ok check to check for the other Buffer package as well would suffice? Or is it deeper and there may be some other incompatibilities in other places?

[daviddias]

I believe that all your issues will be solved if you put at the top of the file where that const ok ... assertion is done a

const Buffer = require('buffer').Buffer

As the module buffer will yield true for buffers created by the Node.js Buffer and itself.

[joaosantos15]

I believe what you're saying is equivalent to this,

var Buffer2 = require('buffer/').Buffer

var newBuffer = Buffer.from('hello')

console.log(Buffer2.isBuffer(newBuffer))

And it still yields false.

update
I tried changing the src/files/add.jsto,

const Buffer = require('buffer').Buffer
    const ok = Buffer.isBuffer(_files) ||
               isStream.readable(_files) ||
               Array.isArray(_files)

And got the same error.

[joaosantos15]

@diasdavid
I explicitly required Buffer, but an getting another error.
src/files/add.js:

...
    const Buffer = require('buffer/').Buffer
    const ok = Buffer.isBuffer(_files) ||
               isStream.readable(_files) ||
               Array.isArray(_files)
...

I'm using this to test:

var ipfsAPI = require('./src/index.js')
var ipfs = ipfsAPI('/ip4/127.0.0.1/tcp/5001')

var Buffer = require('buffer/').Buffer

var myBuffer = Buffer.from('hello')

ipfs.files.add(myBuffer, function (err, result) {
  if (err) {
    console.log(err)
  } else {
    console.log(result[0].hash)
  }
})

Output:

TypeError: content.once is not a function
    at Multipart._pushFile (/git/ipfs/js-ipfs-api/src/utils/multipart.js:80:13)
    at Multipart._maybeDrain (/git/ipfs/js-ipfs-api/src/utils/multipart.js:48:14)
    at Multipart._transform (/git/ipfs/js-ipfs-api/src/utils/multipart.js:40:10)
    at Multipart.Transform._read (_stream_transform.js:167:10)
    at Multipart.Transform._write (_stream_transform.js:155:12)
    at doWrite (_stream_writable.js:307:12)
    at writeOrBuffer (_stream_writable.js:293:5)
    at Multipart.Writable.write (_stream_writable.js:220:11)
    at eachSeries (/git/ipfs/js-ipfs-api/src/utils/send-files-stream.js:52:35)
    at /git/ipfs/js-ipfs-api/node_modules/async/internal/withoutIndex.js:9:16

[daviddias]

@joaosantos15 This is an fairly easy reproducible case, better adding a failing test to the test suite as a PR so that the fix can be worked from there, rather than sharing snippets of code and repro cases.

[alanshaw]

I'm getting this same error when using files.add. Looking at Multipart._pushFile from the stack trace, it looks as though it's assuming the buffer is a stream.

The Buffer.isBuffer check is failing, but it is correct to be failing, because content is not a buffer, it's an object!:

I'll dig a bit deeper and hopefully submit a PR.

[alanshaw]

I've submitted #666 which is for a slightly different issue that causes the same error.

I'm not certain there is an issue here, above some better validation that could be done on the inputs. @joaosantos15, you need to not use require('buffer/').Buffer and just use the global Buffer object.

I think require('buffer/') is giving you a buffer object from your node_modules. This buffer is designed to work in browsers:

The buffer module from node.js, for the browser.

...and it's isBuffer check won't detect a node buffer and vice versa.

In node, when you pass a browser buffer to files.add it uses the node isBuffer to check your buffer, and it fails.

When you pass an array of browser buffer it passes the check that your single buffer failed, because the check only checks for the presence of an array...but actually this is invalid input - if you pass an array it has to contain objects like { path, content } (see spec).

An array of browser buffer gets further but still fails, because the same check happens in Multipart._pushFile - node isBuffer tries to detect a browser buffer and fails. So the code thinks it's a stream that it can attach events to, hence content.once is not a function.

Hope that helps and that this issue can be closed.