Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

size_limit option in copy #6653

Closed
youngsofun opened this issue Jul 15, 2022 · 2 comments
Closed

size_limit option in copy #6653

youngsofun opened this issue Jul 15, 2022 · 2 comments
Labels
A-query Area: databend query

Comments

@youngsofun
Copy link
Member

Summary

multi files load in parallel
row-based counting makes it harder.

@BohuTANG
Copy link
Member

Can we change the parallel to serialized when the size_limit is setting?

@BohuTANG BohuTANG added the A-query Area: databend query label Jul 16, 2022
@youngsofun
Copy link
Member Author

youngsofun commented Jul 17, 2022

maybe we can change a shard remain-rows each time a file/split/block is generated when parse in parallel. ( like push-down)
and add a resize pipeline to 1, to do the final size_limit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-query Area: databend query
Projects
None yet
Development

No branches or pull requests

2 participants