Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Orphan files clean delete files unexpected #877

Closed
1 task done
wangtaohz opened this issue Dec 7, 2022 · 1 comment · Fixed by #878
Closed
1 task done

[Bug]: Orphan files clean delete files unexpected #877

wangtaohz opened this issue Dec 7, 2022 · 1 comment · Fixed by #878
Labels
type:bug Something isn't working

Comments

@wangtaohz
Copy link
Contributor

What happened?

After set 'clean-orphan-file.enabled' = 'true', all manifestList and manifest files will be deleted if the table are not changed for two days.

Affects Versions

master/0.3.0

What engines are you seeing the problem on?

AMS

How to reproduce

  1. create an Arctic table, and write some data
  2. set 'clean-orphan-file.enabled' = 'true', clean-orphan-file.min-existing-time-minutes` = '1'
  3. wait for several minutes
  4. all manifestList files, manifest files and parquet files will be removed

Relevant log output

No response

Anything else

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@wangtaohz wangtaohz added the type:bug Something isn't working label Dec 7, 2022
wangtaohz added a commit to wangtaohz/amoro that referenced this issue Dec 7, 2022
@wangtaohz
fix apache#877 orphan files clean with file uri
b284551
@wangtaohz wangtaohz mentioned this issue Dec 7, 2022
Merged
3 tasks
zhoujinsong pushed a commit that referenced this issue Dec 8, 2022
@wangtaohz
[ARCTIC-877] Fix Orphan files clean delete files unexpected (#878)
4b59f12 
* fix #877 orphan files clean with file uri

* refactor orphan files clean metadata files

* fix checkStyle

* add unit test
wangtaohz added a commit to wangtaohz/amoro that referenced this issue Dec 8, 2022
@wangtaohz
cherry-pick to 0.3.x [ARCTIC-877] Fix Orphan files clean delete files…
f163ead 
… unexpected (apache#878)

* fix apache#877 orphan files clean with file uri

* refactor orphan files clean metadata files

* fix checkStyle

* add unit test
@wangtaohz wangtaohz mentioned this issue Dec 8, 2022
Merged
3 tasks
zhoujinsong pushed a commit that referenced this issue Dec 8, 2022
@wangtaohz
[ARCTIC-877] Cherry-pick to 0.3.x, fix Orphan files clean delete file…
75ea452 
…s unexpected (#882)

cherry-pick to 0.3.x [ARCTIC-877] Fix Orphan files clean delete files unexpected (#878)

* fix #877 orphan files clean with file uri

* refactor orphan files clean metadata files

* fix checkStyle

* add unit test
@wangtaohz
Copy link
Contributor Author

Similar problem in iceberg community Fixes RemoveOrphanFiles delete files unexpected

zhoujinsong pushed a commit that referenced this issue May 31, 2023
@wangtaohz
[ARCTIC-877] Fix Orphan files clean delete files unexpected (#878)
a6d66c4 
* fix #877 orphan files clean with file uri

* refactor orphan files clean metadata files

* fix checkStyle

* add unit test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type:bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[ARCTIC-877] Fix Orphan files clean delete files unexpected wangtaohz/amoro
1 participant
@wangtaohz