pipe works but through doesn't?

[ronag]

Using the following library https://github.com/shovon/bpg-stream

This works but doesn't propagate errors:

_(...)
  // .on('error', ...) // We don't want this...
  .pipe(new BPGStream())
  .on('error', ...)
  .pipe(res)

While this never ends/finishes:

_(...)
  .through(new BPGStream())
  .on('error', ...)
  .pipe(res)

What further requirements does through have over pipe?

[jeromew]

Does the second version start at all ?

The implementations of though and pipe are totally different

• pipe - https://github.com/caolan/highland/blob/2.6.0/lib/index.js#L951 , consumes the source and writes to the target. It returns target
• through - https://github.com/caolan/highland/blob/2.6.0/lib/index.js#L2840 - it pauses the target, and uses the implementation of pipe to return src.pipe(target).pipe(_())

pipe does not forward errors to behave like a standard Readable.

In the through case it would be interesting to understand why it never ends. I suppose this is an interop issue with the way BPGStream ends the stream. Maybe you can activate the debug mode of BPGStream to see if https://github.com/shovon/bpg-stream/blob/master/index.js#L208 is reached.

I try to avoid mixin Node streams with highland streams because there seem to be hard to debug edge cases (interop with node streams is hard).

I have had some success using barrage to go from the highland world to the node stream world or to the Promise world

In your case that would be

npm install barrage

var h = _(...)
barrage(h)
.syphon(new BPGStream())
.on('error', ...)
.pipe(res)

[ronag]

Does the second version start at all ?

I don't quite know. I know it doesn't end.

pipe does not forward errors to behave like a standard Readable.

Yes, I've understood this. Which is why I'd like to use through

I have had some success using barrage to go from the highland world to the node stream world or to the Promise world

I'll try that. Thanks!

[ronag]

Tried this as well without success.

  const out = _()
  const stdout = _(...)
    .on('error', err => out.write(err))
    .pipe(new BPGStream())
    .on('error', err => out.write(err))
    .pipe(out)

[vqvu]

Highland's pipe implementation doesn't emit('pipe') (this is probably a bug). I wonder if that's the problem... Though if it was, then I don't understand why your first example using only pipe works.

[ronag]

@vque: That's because my first example is wrong. My apologies, it should be:

_(...
  // .on('error', ...) // We don't want this...
  .pipe(new BPGStream()))
  .on('error', ...)
  .pipe(res)

[vqvu]

That's it then. Looks like bpg-stream listens for pipe so that it can do on('end') on the source stream, which is kind of silly since the source ending doesn't mean the internal fs stream that bpg-stream uses has finished writing.

Can you try running this?

_(...).through(s => {
        const bpgStream = new BPGStream()
        const res = s.through(bpgStream);
        bpgStream.emit('pipe', s);
        return res;
    })
    .pipe(...)

I tried to do it myself, but I don't have bpgenc installed.

[ronag]

@vqvu: Yes, that solved it.