HDDS-10897. Refactor OzoneQuota #6714
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
szetszwo
reviewed
May 22, 2024
There was a problem hiding this comment.
@adoroszlai , thanks for working on this! Just have two minor suggests.
Comment on lines
43
to
44
| TB(OzoneConsts.TB); | ||
|
|
There was a problem hiding this comment.
Let's add also
+ PB(OzoneConsts.PB),
+ EB(OzoneConsts.EB);
// need to add OzoneConsts.PB and OzoneConsts.EBIt sounds stupid if someone wants to set 1PB quote but Ozone does not support it.
Also, after the above change, the following will never have ArrayIndexOutOfBoundsException.
//RawQuotaInBytes.valueOf(..)
final int i = Long.numberOfTrailingZeros(quotaInBytes) / 10;
final Units unit = Units.values()[i];| /** | ||
| * Used to convert user input values into bytes such as: 1MB-> 1048576. | ||
| */ | ||
| private static class RawQuotaInBytes { | ||
| private Units unit; | ||
| private long size; | ||
| static RawQuotaInBytes valueOf(long quotaInBytes) { |
There was a problem hiding this comment.
Add a precondition:
Preconditions.assertTrue(quotaInBytes >= 0, () -> "quotaInBytes = " + quotaInBytes + " must be >= 0");
szetszwo
approved these changes
May 22, 2024
|
Thanks @szetszwo for reviewing and merging this. |
jojochuang
pushed a commit
to jojochuang/ozone
that referenced
this pull request
May 23, 2024
errose28
added a commit
to errose28/ozone
that referenced
this pull request
May 28, 2024
…concile-cli * HDDS-10239-container-reconciliation: (296 commits) HDDS-10897. Refactor OzoneQuota (apache#6714) HDDS-10422. Fix some warnings about exposing internal representation in hdds-common (apache#6351) HDDS-10899. Refactor Lease callbacks (apache#6715) HDDS-10890. Increase default value for hdds.container.ratis.log.appender.queue.num-elements (apache#6711) HDDS-10832. Client should switch to streaming based on OpenKeySession replication (apache#6683) HDDS-10435. Support S3 object tags for existing requests (apache#6607) HDDS-10883. Improve logging in Recon for finalising DN logic. (apache#6704) HDDS-8752. Enable TestOzoneRpcClientAbstract#testOverWriteKeyWithAndWithOutVersioning (apache#6702) HDDS-10875. XceiverRatisServer#getRaftPeersInPipeline should be called before XceiverRatisServer#removeGroup (apache#6696) HDDS-10514. Recon - Provide DN decommissioning detailed status and info inline with current CLI command output. (apache#6376) HDDS-10878. Bump zstd-jni to 1.5.6-3 (apache#6701) HDDS-10877. Bump Dropwizard metrics to 3.2.6 (apache#6699) HDDS-10876. Bump jackson to 2.16.2 (apache#6697) HDDS-6116. Remove flaky tag from TestSCMInstallSnapshot (apache#6695) HDDS-2643. TestOzoneDelegationTokenSecretManager#testRenewTokenFailureRenewalTime fails intermittently. HDDS-10699. Refactor ContainerBalancerTask and TestContainerBalancerTask (apache#6537) HDDS-10861. Ozone cli supports default ozone.om.service.id (apache#6680) HDDS-10859. Improve error messages when decommission and maintenance fail-early (apache#6678) HDDS-9031. Upgrade acceptance tests to Docker Compose v2 (apache#6667) HDDS-10559. Add a warning or a check to run repair tool as System user (apache#6574) ... Conflicts: hadoop-ozone/dist/src/main/smoketest/admincli/container.robot
szetszwo
reviewed
Jun 4, 2024
Comment on lines
+90
to
+97
| static RawQuotaInBytes valueOf(long quotaInBytes) { | ||
| Preconditions.assertTrue(quotaInBytes >= 0, () -> "quotaInBytes = " + quotaInBytes + " must be >= 0"); | ||
| final int i = Long.numberOfTrailingZeros(quotaInBytes) / 10; | ||
| final Units unit = Units.values()[i]; | ||
| final RawQuotaInBytes b = unit.getRawQuotaInBytes(quotaInBytes >> (i * 10)); | ||
| Preconditions.assertSame(quotaInBytes, b.sizeInBytes(), "sizeInBytes"); | ||
| return b; | ||
| } |
There was a problem hiding this comment.
@adoroszlai , Just found that valueOf(0) will use EB. (Fortunately, we added EB; otherwise, it becomes an ArrayIndexOutOfBoundsException). Not a big deal, although we probably should fix it.
public static void main(String[] args) {
final RawQuotaInBytes q = RawQuotaInBytes.valueOf(0);
System.out.println("q = " + q);
}
// q = 0 EB
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
OzoneQuotaandQuotaListexpose too much state.QuotaListis also inefficient.Extracted from #6351. Most of this change is from review suggestion by @szetszwo.
https://issues.apache.org/jira/browse/HDDS-10897
How was this patch tested?
CI:
https://github.com/adoroszlai/ozone/actions/runs/9189513741