Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deprecate public class names with master terminology #3871

Merged
merged 15 commits into from
Jul 14, 2022

Conversation

tlfeng
Copy link
Collaborator

@tlfeng tlfeng commented Jul 12, 2022

Description

A step of replacing master terminology by cluster manager in the public Java APIs to support inclusive language.

  • Add back the usage of old names for some classes for backwards compatibility. Those public classes were renamed from "master" to "cluster manager" In PR Rename public classes with 'Master' to 'ClusterManager' #3619 / commit a7e113a.
  • Add a unit test to validate the backwards compatibility of interface LocalNodeMasterListener remains.
  • Revert the renaming of class MasterService, FakeThreadPoolMasterService, BlockMasterServiceOnMaster and BusyMasterServiceDisruption, as well as instance variable names of class MasterService.
    Because I couldn't solve the compatibility problem of a public method public ClusterManagerService getMasterService() (Deprecate public class names with master terminology #3871 (comment))

Note:

  1. The interface LocalNodeClusterManagerListener(new name) / LocalNodeMasterListener (old name) is used as the return value for the public method newHashPublisher(). Since LocalNodeClusterManagerListener is the superclass of LocalNodeMasterListener, changing the return value to a superclass will be a breaking change, it will lead to down-casting for others who assigning the return value to a variable, so I changed the return value back to the old class name LocalNodeMasterListener.
  2. Some class path was removed by mistake during the rename refactoring in PR Rename public classes with 'Master' to 'ClusterManager' #3619, I added them back in this PR.
  3. A test in ExceptionSerializationTests requires all subclass of OpenSearchException must be registered in the Exception list (code: https://github.com/opensearch-project/OpenSearch/blob/2.1.0/server/src/test/java/org/opensearch/ExceptionSerializationTests.java#L237). I modified the test to ignore the deprecated exception classes NotMasterException and MasterNotDiscoveredException, since there are new classes NotClusterManagerException and ClusterManagerNotDiscoveredException as the replacement.

List of classes that renamed in previous PR:
In this PR, the usages of the old names should all be restored.
sever directory:
interface LocalNodeMasterListener -> LocalNodeClusterManagerListener
final class MasterNodeChangePredicate -> ClusterManagerNodeChangePredicate
NotMasterException -> NotClusterManagerException
NoMasterBlockService -> NoClusterManagerBlockService
UnsafeBootstrapMasterCommand - UnsafeBootstrapClusterManagerCommand
MasterNotDiscoveredException -> ClusterManagerNotDiscoveredException
RestMasterAction -> RestClusterManagerAction

List of classes that not renamed:
sever directory:
MasterService
test/framework directory:
FakeThreadPoolMasterService
BlockMasterServiceOnMaster
BusyMasterServiceDisruption

Issues Resolved

The second step to resolve issue #3543

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@tlfeng tlfeng requested review from a team and reta as code owners July 12, 2022 22:45
@tlfeng tlfeng added enhancement Enhancement or improvement to existing feature or request v3.0.0 Issues and PRs related to version 3.0.0 backport 2.x Backport to 2.x branch v2.2.0 labels Jul 12, 2022
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@tlfeng
Copy link
Collaborator Author

tlfeng commented Jul 12, 2022

In build 479:


REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.ExceptionSerializationTests.testExceptionRegistration" -Dtests.seed=1E2F54087F845651 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=ru -Dtests.timezone=Asia/Novosibirsk -Druntime.java=17

org.opensearch.ExceptionSerializationTests > testExceptionRegistration FAILED
    java.lang.AssertionError: Classes subclassing OpenSearchException must be registered 
    [class org.opensearch.cluster.NotMasterException, class org.opensearch.discovery.MasterNotDiscoveredException]
        at __randomizedtesting.SeedInfo.seed([1E2F54087F845651:730B14BF0D7650B1]:0)
        at org.junit.Assert.fail(Assert.java:89)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.opensearch.ExceptionSerializationTests.testExceptionRegistration(ExceptionSerializationTests.java:240)

Tianli Feng added 2 commits July 12, 2022 16:39
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@tlfeng
Copy link
Collaborator Author

tlfeng commented Jul 13, 2022

In build 486:

FAILURE: Build completed with 2 failures.

1: Task failed with an exception.
-----------
* What went wrong:
Java heap space
...

…ist in ExceptionSerializationTests

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov-commenter
Copy link

codecov-commenter commented Jul 13, 2022

Codecov Report

Merging #3871 (951af92) into main (23004bd) will decrease coverage by 0.62%.
The diff coverage is 63.21%.

@@             Coverage Diff              @@
##               main    #3871      +/-   ##
============================================
- Coverage     71.10%   70.47%   -0.63%     
+ Complexity    57060    56663     -397     
============================================
  Files          4557     4567      +10     
  Lines        272681   272709      +28     
  Branches      40038    40038              
============================================
- Hits         193884   192193    -1691     
- Misses        62773    64299    +1526     
- Partials      16024    16217     +193     
Impacted Files Coverage Δ
...erver/src/main/java/org/opensearch/Assertions.java 75.00% <ø> (ø)
.../main/java/org/opensearch/OpenSearchException.java 93.03% <ø> (+0.20%) ⬆️
...n/cluster/health/TransportClusterHealthAction.java 47.75% <0.00%> (ø)
...min/cluster/state/TransportClusterStateAction.java 53.94% <ø> (ø)
.../opensearch/cluster/MasterNodeChangePredicate.java 0.00% <0.00%> (-90.91%) ⬇️
...ava/org/opensearch/cluster/NotMasterException.java 0.00% <ø> (-100.00%) ⬇️
...rch/cluster/coordination/NoMasterBlockService.java 0.00% <0.00%> (-100.00%) ⬇️
...g/opensearch/cluster/coordination/NodeToolCli.java 0.00% <0.00%> (ø)
...dination/UnsafeBootstrapClusterManagerCommand.java 0.00% <0.00%> (ø)
...ter/coordination/UnsafeBootstrapMasterCommand.java 0.00% <0.00%> (ø)
... and 515 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7e113a...951af92. Read the comment docs.

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@tlfeng
Copy link
Collaborator Author

tlfeng commented Jul 13, 2022

Note: The above build 483 didn't run the test for the latest commit, so it can be ignored.

Tianli Feng added 2 commits July 13, 2022 15:06
…erListener

Signed-off-by: Tianli Feng <ftianli@amazon.com>
…eprecated master setting

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@tlfeng
Copy link
Collaborator Author

tlfeng commented Jul 13, 2022

In build 559:

  2> REPRODUCE WITH: ./gradlew ':server:internalClusterTest' --tests "org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark" -Dtests.seed=12AEA4311233E792 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es-PA -Dtests.timezone=Asia/Aqtau -Druntime.java=17
  2> java.lang.AssertionError: Mismatching shard routings: [[yqylmgkorg][4], node[cG3A-2D6Rse374wO2ZuF-A], [P], s[STARTED], a[id=8eWSfYO9QaWXv7vcZc1DZw], [yqylmgkorg][5], node[cG3A-2D6Rse374wO2ZuF-A], [P], s[STARTED], a[id=pZzhjFI0Q1ifCSXLzeeaFw]]
    Expected: a collection with size <1>
         but: collection size was <2>
        at __randomizedtesting.SeedInfo.seed([12AEA4311233E792:160250798AF82A]:0)
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
        at org.junit.Assert.assertThat(Assert.java:964)
        at org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.lambda$assertBusyWithDiskUsageRefresh$5(DiskThresholdDeciderIT.java:363)
        at org.opensearch.test.OpenSearchTestCase.assertBusy(OpenSearchTestCase.java:1049)
        at org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.assertBusyWithDiskUsageRefresh(DiskThresholdDeciderIT.java:356)
        at org.opensearch.cluster.routing.allocation.decider.DiskThresholdDeciderIT.testRestoreSnapshotAllocationDoesNotExceedWatermark(DiskThresholdDeciderIT.java:275)

The below is reported in issue #3872

REPRODUCE WITH: ./gradlew ':server:test' --tests "org.opensearch.indices.replication.SegmentReplicationTargetServiceTests.testShardAlreadyReplicating" -Dtests.seed=12AEA4311233E792 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en-GB -Dtests.timezone=America/Swift_Current -Druntime.java=17

org.opensearch.indices.replication.SegmentReplicationTargetServiceTests > testShardAlreadyReplicating FAILED
    java.lang.AssertionError: can't move replication to stage [DONE]. current stage: [INIT] (expected [REPLICATING])
        at __randomizedtesting.SeedInfo.seed([12AEA4311233E792:3177212A98F42E9B]:0)
        at org.opensearch.indices.replication.SegmentReplicationState.validateAndSetStage(SegmentReplicationState.java:88)
        at org.opensearch.indices.replication.SegmentReplicationState.setStage(SegmentReplicationState.java:107)
        at org.opensearch.indices.replication.SegmentReplicationTarget.onDone(SegmentReplicationTarget.java:87)
        at org.opensearch.indices.replication.common.ReplicationTarget.markAsDone(ReplicationTarget.java:143)
        at org.opensearch.indices.replication.SegmentReplicationTargetServiceTests.testShardAlreadyReplicating(SegmentReplicationTargetServiceTests.java:139)

@tlfeng
Copy link
Collaborator Author

tlfeng commented Jul 13, 2022

Hi @reta, thanks a lot for your review! It really helps me push the process forward!

Comment on lines 44 to 47
// TODO: add final keyword to the class and private keyword to the default constructor,
// after removing the deprecated class MasterNodeChangePredicate.
// Removed the final keyword temporarily only for making the class MasterNodeChangePredicate as a subclass,
// so that preserving the both class names by maintaining one class implementation for backwards compatibility.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this class only exists as a container for a single static method. Assuming that's right, the less intrusive approach may be just to do the following in the other class (as opposed to using inheritance):

@Deprecated
public final class MasterNodeChangePredicate {
  private MasterNodeChangePredicate() { }

  public static Predicate<ClusterState> build(ClusterState currentState) {
    return ClusterManagerNodeChangePredicate.build(clusterState);
  }
}

Because it means you don't have to change this class at all.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you so much for providing the suggestion to me! Let me check this approach. 😄

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This must be the "delegation" approach. Thanks for showing me the code. I updated in the commit d10fec2

…sterService to keep backwards compatibility

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

public MasterService(Settings settings, ClusterSettings clusterSettings, ThreadPool threadPool) {
super(settings, clusterSettings, threadPool);
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is one problem, the return value of getMasterService() should be the old MasterService class, otherwise there will be down-casting problem for the existing classes that uses the return value.
https://github.com/tlfeng/OpenSearch/blob/951af92cb316bf0d9f1b56338fd0bec90d029375/server/src/main/java/org/opensearch/cluster/service/ClusterService.java#L221

    public ClusterManagerService getMasterService() {
        return clusterManagerService;
    }

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Finally I reverted renaming the class MasterService in this PR (in commit 50a322b).

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

This reverts commit 694d876.

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@tlfeng tlfeng marked this pull request as draft July 14, 2022 06:06
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Tianli Feng added 3 commits July 14, 2022 13:52
Signed-off-by: Tianli Feng <ftianli@amazon.com>
Signed-off-by: Tianli Feng <ftianli@amazon.com>
…eDisruption

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

Signed-off-by: Tianli Feng <ftianli@amazon.com>
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

tlfeng pushed a commit that referenced this pull request Jul 14, 2022
#3870)

* Rename public classes with 'Master' to 'ClusterManager' (#3619)

Replace master terminology by cluster manager in the public Java APIs to support inclusive language.
The PR deal with the public class name in the repository (except for those covered in issue #3542).

* Replace `Master` to `ClusterManager` in most of the class names (except `MasterService` class), including all the references to the classes.

The next PR will be #3871, adding back the classes in old name for backwards compatibility.

List of classes that renamed in this PR:
`sever` directory:
interface LocalNodeMasterListener -> LocalNodeClusterManagerListener
final class MasterNodeChangePredicate -> ClusterManagerNodeChangePredicate
NotMasterException -> NotClusterManagerException
NoMasterBlockService -> NoClusterManagerBlockService
UnsafeBootstrapMasterCommand - UnsafeBootstrapClusterManagerCommand
MasterNotDiscoveredException -> ClusterManagerNotDiscoveredException
RestMasterAction -> RestClusterManagerAction

List of classes that not renamed:
`sever` directory:
MasterService
`test/framework` directory:
FakeThreadPoolMasterService
BlockMasterServiceOnMaster
BusyMasterServiceDisruption

Signed-off-by: Tianli Feng <ftianli@amazon.com>
(cherry picked from commit a7e113a)
@tlfeng tlfeng marked this pull request as ready for review July 14, 2022 23:38
@tlfeng tlfeng merged commit 0e96a87 into opensearch-project:main Jul 14, 2022
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-3871-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0e96a878a5ab1ffec3ee04f15fe976d34c9905dc
# Push it to GitHub
git push --set-upstream origin backport/backport-3871-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-3871-to-2.x.

@tlfeng tlfeng deleted the deprecate-master-class branch July 14, 2022 23:40
tlfeng pushed a commit to tlfeng/OpenSearch that referenced this pull request Jul 15, 2022
…ect#3871)

- Add back the usage of old names for some classes for backwards compatibility. Those public classes were renamed from "master" to "cluster manager" In PR opensearch-project#3619 / commit opensearch-project@a7e113a.
 - Add a unit test to validate the backwards compatibility of interface `LocalNodeMasterListener` remains.
 - Revert the renaming of class `MasterService`, `FakeThreadPoolMasterService`, `BlockMasterServiceOnMaster` and `BusyMasterServiceDisruption`, as well as instance variable names of class `MasterService`.
 Because I couldn't solve the compatibility problem of a public method `public ClusterManagerService getMasterService()` (opensearch-project#3871 (comment))

**List of classes that renamed in previous PR:**
In this PR, the usages of the old names should all be restored.
`sever` directory:
interface LocalNodeMasterListener -> LocalNodeClusterManagerListener
final class MasterNodeChangePredicate -> ClusterManagerNodeChangePredicate
NotMasterException -> NotClusterManagerException
NoMasterBlockService -> NoClusterManagerBlockService
UnsafeBootstrapMasterCommand - UnsafeBootstrapClusterManagerCommand
MasterNotDiscoveredException -> ClusterManagerNotDiscoveredException
RestMasterAction -> RestClusterManagerAction

**List of classes that not renamed:**
`sever` directory:
MasterService
`test/framework` directory:
FakeThreadPoolMasterService
BlockMasterServiceOnMaster
BusyMasterServiceDisruption

Signed-off-by: Tianli Feng <ftianli@amazon.com>
tlfeng pushed a commit that referenced this pull request Jul 15, 2022
…3914)

* Deprecate public class names with master terminology (#3871)

- Add back the usage of old names for some classes for backwards compatibility. Those public classes were renamed from "master" to "cluster manager" In PR #3619 / commit a7e113a.
 - Add a unit test to validate the backwards compatibility of interface `LocalNodeMasterListener` remains.
 - Revert the renaming of class `MasterService`, `FakeThreadPoolMasterService`, `BlockMasterServiceOnMaster` and `BusyMasterServiceDisruption`, as well as instance variable names of class `MasterService`.
 Because I couldn't solve the compatibility problem of a public method `public ClusterManagerService getMasterService()` (#3871 (comment))

**List of classes that renamed in previous PR:**
In this PR, the usages of the old names should all be restored.
`sever` directory:
interface LocalNodeMasterListener -> LocalNodeClusterManagerListener
final class MasterNodeChangePredicate -> ClusterManagerNodeChangePredicate
NotMasterException -> NotClusterManagerException
NoMasterBlockService -> NoClusterManagerBlockService
UnsafeBootstrapMasterCommand - UnsafeBootstrapClusterManagerCommand
MasterNotDiscoveredException -> ClusterManagerNotDiscoveredException
RestMasterAction -> RestClusterManagerAction

**List of classes that not renamed:**
`sever` directory:
MasterService
`test/framework` directory:
FakeThreadPoolMasterService
BlockMasterServiceOnMaster
BusyMasterServiceDisruption

Signed-off-by: Tianli Feng <ftianli@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch deprecate enhancement Enhancement or improvement to existing feature or request v2.2.0 v3.0.0 Issues and PRs related to version 3.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants