HDDS-10231. ContainerStateManager should not finalize the OPEN containers without a Pipeline. #6123
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ners without a Pipeline.
sodonnel
reviewed
Jan 30, 2024
...t/java/org/apache/hadoop/hdds/scm/container/replication/health/TestOpenContainerHandler.java
Show resolved
Hide resolved
sodonnel
reviewed
Jan 30, 2024
...t/java/org/apache/hadoop/hdds/scm/container/replication/health/TestOpenContainerHandler.java
Show resolved
Hide resolved
|
Looks mostly good. I just have a suggestion to add another check in a couple of the tests. |
|
Thanks for the review @sodonnel. |
siddhantsangwan
approved these changes
Jan 30, 2024
|
Thanks @nandakumar131 for the patch, @siddhantsangwan, @sodonnel for the review. |
|
@nandakumar131 Warn logs needs to be updated as telling it "marking container for closing", but this logic is moved now to Replication manager |
|
@sumitagrawl, thanks for the review. I will update the log in one of my following PRs. |
myskov
pushed a commit
to myskov/ozone
that referenced
this pull request
Apr 3, 2024
… without a Pipeline. (apache#6123) (cherry picked from commit 04d6e0b)
myskov
pushed a commit
to myskov/ozone
that referenced
this pull request
Apr 3, 2024
… without a Pipeline. (apache#6123) (cherry picked from commit 04d6e0b)
myskov
pushed a commit
to myskov/ozone
that referenced
this pull request
Apr 3, 2024
… without a Pipeline. (apache#6123) (cherry picked from commit 04d6e0b)
swamirishi
pushed a commit
to swamirishi/ozone
that referenced
this pull request
Jun 10, 2024
…N containers without a Pipeline. (apache#6123) (cherry picked from commit 04d6e0b) Change-Id: If6fa9aa4447c7664e0a4f9794244af4711fc2ca6
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
If there is an OPEN container without any pipeline, that container has to be moved to CLOSING state and CLOSE container command has to be sent to the datanodes.
Currently, this is done while SCM is initialized. Since Ratis is not set up completely (mostly no leader is elected at this point), the state update is done locally (not via Ratis).
Doing this on SCM start-up is error-prone.
The container state update has to be done only via Ratis. If not, there are chances of container state getting diverged between SCMs.
This PR changes the logic and moves it to ReplicationManager's
OpenContainerHandler, where we check if the OPEN Container has a healthy Pipeline, if not the container is closed.What is the link to the Apache JIRA
HDDS-10231
How was this patch tested?
Added new unit test to cover the scenario.