Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve] Add num not adhering placement ledgers replicated metric for ReplicationWorker #3652

Conversation

wenbingshen
Copy link
Member

Motivation

We have NUM_FULL_OR_PARTIAL_LEDGERS_REPLICATED in ReplicationWorker, which includes both DATA_LOSS and DATA_NOT_ADHERING_PLACEMENT data repair types. It is meaningful to add NUM_NOT_ADHERING_PLACEMENT_LEDGERS_REPLICATED metric for DATA_NOT_ADHERING_PLACEMENT type separately.

Copy link
Member

@zymap zymap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall look good to me. Just need a test.

@@ -1262,6 +1268,9 @@ protected EnsemblePlacementPolicy initializeEnsemblePlacementPolicy(ClientConfig
assertNull(stat1);
});

assertTrue("NUM_NOT_ADHERING_PLACEMENT_LEDGERS_REPLICATED",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a new test to make sure the number is increased as expected.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good suggestion. I found a bug about numLedgersReplicated update. I have submit another PR #3654 to fix it. PTAL. I will update this PR when that PR is merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@hangc0276
Copy link
Contributor

@horizonzy Please help take a look at this PR, thanks.

@horizonzy
Copy link
Member

@horizonzy Please help take a look at this PR, thanks.

Fine

Copy link
Contributor

@hangc0276 hangc0276 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LTGM, thanks for your contribution.

@@ -1262,6 +1268,9 @@ protected EnsemblePlacementPolicy initializeEnsemblePlacementPolicy(ClientConfig
assertNull(stat1);
});

assertTrue("NUM_NOT_ADHERING_PLACEMENT_LEDGERS_REPLICATED",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Copy link
Member

@horizonzy horizonzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@hangc0276 hangc0276 modified the milestones: 4.17.0, 4.16.0 Jan 11, 2023
hangc0276 pushed a commit that referenced this pull request Feb 15, 2023
### Motivation

When I want to add num not adhering placement ledgers replicated metric for RW(#3652), I found the numLedgersReplicated statistic to be inaccurate.

When we could not find a target bookie to replicate the ledger, the numLedgersReplicated metric will still increase.
![image](https://user-images.githubusercontent.com/35599757/202638656-58f974f6-9ffc-4cfe-9ac7-85962f941de7.png)
@codecov-commenter
Copy link

codecov-commenter commented Feb 28, 2023

Codecov Report

Merging #3652 (4f9eeaa) into master (0ca4fe2) will decrease coverage by 8.51%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##             master    #3652      +/-   ##
============================================
- Coverage     68.32%   59.81%   -8.51%     
+ Complexity     6753     5947     -806     
============================================
  Files           473      473              
  Lines         40972    40979       +7     
  Branches       5241     5243       +2     
============================================
- Hits          27993    24511    -3482     
- Misses        10720    14341    +3621     
+ Partials       2259     2127     -132     
Flag Coverage Δ
bookie 39.90% <0.00%> (+0.03%) ⬆️
client 44.24% <42.85%> (+0.03%) ⬆️
remaining 29.74% <100.00%> (+0.04%) ⬆️
replication ?
tls 21.05% <0.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...ache/bookkeeper/replication/ReplicationWorker.java 75.85% <100.00%> (+0.45%) ⬆️
...rg/apache/bookkeeper/metastore/MetastoreTable.java 0.00% <0.00%> (-100.00%) ⬇️
.../bookkeeper/metastore/MetastoreScannableTable.java 0.00% <0.00%> (-100.00%) ⬇️
...bookkeeper/server/http/service/MetricsService.java 0.00% <0.00%> (-100.00%) ⬇️
...eper/server/http/service/ConfigurationService.java 0.00% <0.00%> (-100.00%) ⬇️
...per/meta/LongHierarchicalLedgerManagerFactory.java 0.00% <0.00%> (-100.00%) ⬇️
...ookeeper/ExponentialBackOffWithDeadlinePolicy.java 0.00% <0.00%> (-100.00%) ⬇️
...eper/server/http/service/BookieIsReadyService.java 0.00% <0.00%> (-93.34%) ⬇️
...okkeeper/server/http/service/GCDetailsService.java 0.00% <0.00%> (-90.00%) ⬇️
...eeper/server/http/service/DeleteLedgerService.java 0.00% <0.00%> (-88.89%) ⬇️
... and 150 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@wenbingshen
Copy link
Member Author

rerun failure checks

1 similar comment
@wenbingshen
Copy link
Member Author

rerun failure checks

@wenbingshen
Copy link
Member Author

Hi, @zymap @hangc0276 I have updated this PR. Could you help take a look? Thanks.

@wenbingshen wenbingshen changed the title Add num not adhering placement ledgers replicated metric for ReplicationWorker [improve] Add num not adhering placement ledgers replicated metric for ReplicationWorker Feb 28, 2023
@hangc0276 hangc0276 merged commit 08c3138 into apache:master Mar 7, 2023
Ghatage pushed a commit to sijie/bookkeeper that referenced this pull request Jul 12, 2024
### Motivation

When I want to add num not adhering placement ledgers replicated metric for RW(apache#3652), I found the numLedgersReplicated statistic to be inaccurate.

When we could not find a target bookie to replicate the ledger, the numLedgersReplicated metric will still increase.
![image](https://user-images.githubusercontent.com/35599757/202638656-58f974f6-9ffc-4cfe-9ac7-85962f941de7.png)
Ghatage pushed a commit to sijie/bookkeeper that referenced this pull request Jul 12, 2024
…ionWorker (apache#3652)

### Motivation

We have `NUM_FULL_OR_PARTIAL_LEDGERS_REPLICATED` in `ReplicationWorker`, which includes both `DATA_LOSS` and `DATA_NOT_ADHERING_PLACEMENT` data repair types. It is meaningful to add `NUM_NOT_ADHERING_PLACEMENT_LEDGERS_REPLICATED` metric for `DATA_NOT_ADHERING_PLACEMENT` type separately.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants