Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(csv): CSVArrowLoader #3135

Merged
merged 7 commits into from
Oct 17, 2024
Merged

feat(csv): CSVArrowLoader #3135

merged 7 commits into from
Oct 17, 2024

Conversation

ibgreen
Copy link
Collaborator

@ibgreen ibgreen commented Oct 16, 2024

Adding a CSVArrowLoader that returns arrow formatted data.

  • This is a first simple implementation that just converts the result of the loader to arrow.
  • The atomic load will not be efficient, but batched parsing might be reasonably useful as it converts each batch as it goes and does not hold on the big JS table.

@ibgreen ibgreen requested a review from igorDykhta October 16, 2024 20:54
@ibgreen ibgreen requested a review from heshan0131 October 16, 2024 20:55
@ibgreen ibgreen merged commit 264e413 into master Oct 17, 2024
2 checks passed
@ibgreen ibgreen deleted the ib/csv-arrow-loader branch October 17, 2024 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@igorDykhta igorDykhta Awaiting requested review from igorDykhta

@heshan0131 heshan0131 Awaiting requested review from heshan0131

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@ibgreen @heshan0131