Add support for WebStreams

[2color]

Background

I'm exploring verified (trustless) retrieval of CAR files (See olizilla/ipfs-get#15) in browsers.

I discovered that there's no way to instantiate a CarReader from a ReadableStream returned from the Fetch API in the browser.

This is because Readable Web Streams are not iterable by default.

Adding support for Web Streams

For us to be able to promote verified retrieval in browsers, it would be nice to have full CAR support in browsers. For this, we need a way to instantiate a CAR Reader in the browser from a Readable Web Streams.

My current work around is to use @achingbrain's browser-readablestream-to-it
library

[RangerMauve]

In my experience, one can convert ReadableStreams into async iterators with either polyfills for a built in [Symbol.asyncIterator]() method, or using the example in the mdn docs which can convert to one in code.

Is this something we can add docs in the README for, or is it absolutely necessary to be able to accept ReadableStreams as arguments?

[RangerMauve]

The spec discussion for making them async iterable can be found here: whatwg/streams#778 (comment)

I think the current best practice is to include a polyfill in your code for it since it's slowly making it's way into implementations.

[2color]

or using the example in the mdn docs which can convert to one in code.

That converts it the other way (iterator -> stream)

Is this something we can add docs in the README for, or is it absolutely necessary to be able to accept ReadableStreams as arguments?

That's not for me to decide, but I'd add a note about this, at the very least. Accepting ReadableStreams would be a nice DX improvement, given the primacy of web APIs and their universality.

[BigLep]

2023-02-21 maintainer conversation: IPLD maintenance team is a skeleton crew right now. Is this something you would want to pick up @2color ? @rvagg is happy to review or answer questions.

[2color]

That's understandable.

I don't have the capacity right now. But I'll revisit this if time allows.

[RangerMauve]

Might experiment with this in WebRecorder since we're working on some custom chunking to CAR use cases.