Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: idempotent-copy file #7719

Merged
merged 9 commits into from
Sep 19, 2022
Merged

feat: idempotent-copy file #7719

merged 9 commits into from
Sep 19, 2022

Conversation

lichuang
Copy link
Contributor

@lichuang lichuang commented Sep 19, 2022 •
edited
Loading

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

  1. idempotent-copy file
  2. add force option in copy into SQL.
  3. add truncate_table in catalog to truncate all the copied files info in meta service.
  4. add test for idempotent-copy and force copy.

Fixes #7550, #7561

@vercel
Copy link

vercel bot commented Sep 19, 2022 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

1 Ignored Deployment
Name Status Preview Updated
databend ⬜️ Ignored (Inspect) Sep 19, 2022 at 2:22PM (UTC)

@lichuang lichuang marked this pull request as ready for review September 19, 2022 06:34
@lichuang lichuang requested review from Xuanwo and BohuTANG September 19, 2022 06:34
@mergify mergify bot added the pr-feature this PR introduces a new feature to the codebase label Sep 19, 2022
@lichuang lichuang marked this pull request as draft September 19, 2022 06:54
@lichuang lichuang marked this pull request as ready for review September 19, 2022 07:11
@BohuTANG
Copy link
Member

@mergify update

@mergify
Copy link
Contributor

mergify bot commented Sep 19, 2022

update

✅ Branch has been successfully updated

@BohuTANG
Copy link
Member

runner/_work/databend/databend/tests/suites/1_stateful/00_copy/00_0000_copy_from_stage.stdout 2022-09-19 13:49:19.823196115 +0000
252
@@ -33,4 +33,4 @@
253
597 2020.0 2307
254
5
255
0
256
-398 2020.0 769
257
+398 2020.0 1538

@BohuTANG BohuTANG requested a review from Xuanwo September 19, 2022 15:05
@BohuTANG BohuTANG merged commit 24ea516 into databendlabs:main Sep 19, 2022
@BohuTANG
Copy link
Member

@soyeric128
Please update the copy command, we introduced a new option: FORCE

@lichuang lichuang deleted the idempotent-copy branch September 20, 2022 02:13
@BohuTANG BohuTANG mentioned this pull request Sep 21, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BohuTANG BohuTANG BohuTANG approved these changes

@Xuanwo Xuanwo Awaiting requested review from Xuanwo

Assignees
No one assigned
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Idempotent Copy of stage files into table. @lichuang
3 participants
@lichuang @BohuTANG @Xuanwo