Fix streaming interface to parseCSV #342
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a couple of tiny fixes to support CSV streaming, plus a couple of unit tests to show the expected usage.
Details
There were a couple of wee problems with the streaming API:
response.body
straight intocsv-parse
. response.body is a browser-compliant ReadableStream, but we need it to be a node Readable. Subtle. All we have to do is callReadable.from
to convert the stream to a compatible APIThere are tests in index.test.js which show streaming working from the filesystem and from unidici request. I have also tested this without a mock (ie, host a csv file locally with
npx serve
, pull it down with node native fetch, and parse it) - but obviously there's no use commiting that. I am satisfied that the mock exercises the same API.@mtuchi I think you should probably merge this into your
msgraph
work for akf. You should be able to stream freely from there.