HDFS-17438. RBF: The newest STANDBY and UNAVAILABLE nn should be the lowest priority. #6655
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KeeProMise
changed the title
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
goiri
reviewed
Mar 29, 2024
| try { | ||
| DatanodeInfo[] dns = this.dnCache.get(type); | ||
| if (dns == null) { | ||
| if (dns.length == 0) { |
Member
There was a problem hiding this comment.
I think it is safer to check for both null and length.
...bf/src/test/java/org/apache/hadoop/hdfs/server/federation/resolver/TestNamenodeResolver.java
Show resolved
Hide resolved
KeeProMise
changed the title
|
💔 -1 overall
This message was automatically generated. |
Member
Author
|
@goiri hi, hadoop-hdfs-project/hadoop-hdfs-rbf in trunk has 1 extant spotbugs warnings, do you have any suggestions? |
goiri
reviewed
Apr 1, 2024
| // Verify the selected entry is the oldest standby entry | ||
| assertTrue(namenodeResolver.registerNamenode(createNamenodeReport( | ||
| NAMESERVICES[0], NAMENODES[0], HAServiceState.STANDBY))); | ||
| Thread.sleep(1500); |
Member
There was a problem hiding this comment.
We shouldn't be adding waits like this, can we add a GenericTestUtils#waitFor or similar?
Member
Author
There was a problem hiding this comment.
This suggestion sounds good, I added GenericTestUtils#atLeastWaitFor method, please help to take a look.
|
💔 -1 overall
This message was automatically generated. |
slfan1989
reviewed
Apr 3, 2024
| try { | ||
| return namenodeResolver.registerNamenode( | ||
| createNamenodeReport(nsId, nnId, haServiceState)); | ||
| }catch (IOException e) { |
Contributor
There was a problem hiding this comment.
small comment, There should be a space
Member
Author
There was a problem hiding this comment.
Thank you for your review, done.
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
Member
Author
|
@goiri hi, If you have time, please take a look again, thanks. |
|
💔 -1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
goiri
requested changes
May 12, 2024
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java
Show resolved
Hide resolved
|
💔 -1 overall
This message was automatically generated. |
Member
Author
|
@goiri The failed ut should not be caused by my modifications. |
goiri
approved these changes
May 31, 2024
Member
Author
|
@goiri hi， if no comments here,please help merge it. Thanks! |
Contributor
|
We're closing this stale PR because it has been open for 100 days with no activity. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
seeAlso: https://issues.apache.org/jira/browse/HDFS-17438
At present, when the status of all namenodes in an ns in the router is the same, the namenode which is the newest reported will be placed at the top of the cache. when the client accesses the ns through the router, it will first access the namenode.
If multiple namenodes in this route are in an active state, or if there are namenodes with multiple observer states, the existing logic is not a problem, because the newest reported active or observer state namenode have a higher probability of being true active or observer compared to the namenode that reported active or observer state a long time ago.
Similarly, the newest reported namenode with a status of standby or unavailable has a higher probability of being a standby or unavailable namenode compared to the namenode reported with a status of standby or unavailable a long time ago. Therefore, the newest nn reported as standby or unavailable status should have a lower priority for access, the oldest nn reported as standby or unavailable status should have a higher priority for access.
How was this patch tested?
use TestNamenodeResolver#testRegistrationNamenodeSelection().
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?