Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix][broker] Delete compacted ledger when topic is deleted #21745

Merged
merged 10 commits into from
Dec 29, 2023

Conversation

coderzc
Copy link
Member

@coderzc coderzc commented Dec 18, 2023

Motivation

Currently, the compacted ledger doesn't delete when the topic is deleted, this will result in many orphaned ledgers, we should delete them.

Modifications

Delete compacted ledger when __compaction cursor is deleted.

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Dec 18, 2023
@@ -106,5 +108,16 @@ public void markDeleteFailed(ManagedLedgerException exception, Object ctx) {
}
}

CompletableFuture<Void> cleanCompactedLedger() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have TopicCompactionService now, shall we add an new API to the TopicCompactionService?

Copy link
Member Author

@coderzc coderzc Dec 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR is just to fix this old issue, it needs to be cherry-picked to the release branch, I think we can separately propose a PIP extending TopicCompactionService on the later version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, sure.

@coderzc coderzc force-pushed the fix_delete_compacted_ledger branch from ebe004a to c9d3068 Compare December 27, 2023 09:50
Copy link
Contributor

@BewareMyPower BewareMyPower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition, to keep the same style, you can use assertEquals instead of Assert.assertEquals.

@coderzc coderzc force-pushed the fix_delete_compacted_ledger branch from 66ecc41 to 5f96de3 Compare December 28, 2023 05:20
@coderzc coderzc force-pushed the fix_delete_compacted_ledger branch from 5f96de3 to 3a2bf54 Compare December 28, 2023 05:21
@coderzc coderzc merged commit 23391d3 into apache:master Dec 29, 2023
47 checks passed
@coderzc coderzc deleted the fix_delete_compacted_ledger branch December 29, 2023 06:39
coderzc added a commit to coderzc/pulsar that referenced this pull request Jan 4, 2024
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jan 4, 2024
srinath-ctds pushed a commit to datastax/pulsar that referenced this pull request Jan 8, 2024
coderzc added a commit that referenced this pull request Jan 11, 2024
@coderzc coderzc modified the milestones: 3.3.0, 3.2.0 May 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants