[Improvement]: Optimize can't stop when the change store only has eq-delete files and the base store has no files

[hzluting]

Search before asking

• I have searched in the issues and found no similar issues.

What would you like to be improved?

Even if the minor optimize output is nothing, the commit still should update "max-txId" in partition properties.

How should we improve?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Subtasks

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[zhoujinsong]

It seems this is a complicated situation.
Could you give more detail to help us understand it? like:

• Table types that the problem may affect
• What is the situation the problem occurs
• What do you think causes the problem and how should we fix it

[hzluting]

It seems this is a complicated situation. Could you give more detail to help us understand it? like:

• Table types that the problem may affect
• What is the situation the problem occurs
• What do you think causes the problem and how should we fix it

The problem only occurs in the arctic keyed table. There are no files in the base store and only some eq-delete files in the change store. So the table will trigger minor optimize, but this optimize will output nothing. Then optimize commit will skip, because there are no changes for the base store, so "max-txid" in the base store property won't be updated, and the minor optimize won't stop.
I will add logic to update "max-txid" in the base store property, even if minor optimize doesn't produce any files.

[zhoujinsong]

It seems this is a complicated situation. Could you give more detail to help us understand it? like:

• Table types that the problem may affect
• What is the situation the problem occurs
• What do you think causes the problem and how should we fix it

The problem only occurs in the arctic keyed table. There are no files in the base store and only some eq-delete files in the change store. So the table will trigger minor optimize, but this optimize will output nothing. Then optimize commit will skip, because there are no changes for the base store, so "max-txid" in the base store property won't be updated, and the minor optimize won't stop. I will add logic to update "max-txid" in the base store property, even if minor optimize doesn't produce any files.

Get it. Thanks for your reply.