Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: optimize replace into data block reading #11780

Merged
merged 4 commits into from
Jun 18, 2023

Conversation

dantengsky
Copy link
Member

@dantengsky dantengsky commented Jun 16, 2023

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

while executing replace into, for data blocks that could not be pruned by using range index, this PR introduces an optimization similar to prewhere:

  • first load the block data of columns specified by the on conflict clause, and filter the rows
  • only if rows of the block are partially deleted, we need to load the block data of other columns (if any), to generate the new block.

Closes #issue

@vercel
Copy link

vercel bot commented Jun 16, 2023

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Comments Updated (UTC)
databend ⬜️ Ignored (Inspect) Jun 17, 2023 2:42pm

@mergify mergify bot added the pr-refactor this PR changes the code base without new features or bugfix label Jun 16, 2023
@dantengsky dantengsky marked this pull request as ready for review June 17, 2023 15:04
@dantengsky dantengsky requested review from zhyass and BohuTANG June 17, 2023 15:04
@BohuTANG BohuTANG merged commit 69583de into databendlabs:main Jun 18, 2023
andylokandy pushed a commit to andylokandy/databend that referenced this pull request Nov 27, 2023
* optimize replace into data block reading

* fix sql logic test

* fix sqllogic test

* more sqllogic test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-refactor this PR changes the code base without new features or bugfix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants