Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest #12472

Merged
merged 3 commits into from
Aug 9, 2022

Conversation

yashmayya
Copy link
Contributor

https://issues.apache.org/jira/browse/KAFKA-14134

From https://issues.apache.org/jira/browse/KAFKA-7438:

Development of EasyMock and PowerMock has stagnated while Mockito continues to be actively developed. With the new Java cadence, it's a problem to depend on libraries that do bytecode generation and are not actively maintained. In addition, Mockito is also easier to use.

@yashmayya
Copy link
Contributor Author

@mimaison or @C0urante could you please take a look? The changes are fairly straightforward for this one.

Copy link
Contributor

@divijvaidya divijvaidya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review. I added a minor comment, otherwise looks good.

connector.version();
expectLastCall().andReturn(VERSION);

offsetStore.start();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am adding this comment here to help other reviewers in reviewing the changes in these lines.

It is ok to remove these void calls because Mockito automatically creates stubs for void method calls. As long as we are verifying that they are being invoked, we should be good.

Copy link
Contributor

@C0urante C0urante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Yash. I haven't looked closely at every individual case yet, but I've left a few stylistic remarks. I'm hoping that if we can clean some of the logic here into succinct, reusable methods then it'll be easier to review and verify with certainty that we haven't mistranslated anything or accidentally lost guarantees. Every case I have examined looks good so far though 👍

…verify calls into separate methods; reduce number of SuppressWarning annotations
@yashmayya
Copy link
Contributor Author

Thanks for the reviews @divijvaidya and @C0urante! I've incorporated your feedback, could you please take another look?

@divijvaidya
Copy link
Contributor

Looks good from my side!

Copy link
Contributor

@C0urante C0urante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking pretty good! Just a few more comments.

verifyAll();
verifyCleanInitialize();
verify(listener).onFailure(CONNECTOR, exception);
verify(listener).onShutdown(CONNECTOR);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Almost every call to verifyCleanShutdown is preceded by a check to verify that listener::onShutdown was invoked. Could we add this logic to verifyCleanShutdown? Thinking of something like this:

private void verifyCleanShutdown() {
    verifyShutdown(true);
}

private void verifyShutdown(boolean clean) {
    verify(ctx).close();
    verify(offsetStorageReader).close();
    verify(offsetStore).stop();
    if (clean) {
        verify(listener).onShutdown(CONNECTOR);
    }
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can also follow a similar approach with startup logic. This would involve renaming verifyCleanInitialize to verifyStartup and adding a clean parameter to the method:

private void verifyCleanStartup() {
    verifyStartup(true);
}

private void verifyStartup(boolean clean) {
    verify(connector).version();
    if (connector instanceof SourceConnector) {
        verify(offsetStore).start();
        verify(connector).initialize(any(SourceConnectorContext.class));
    } else {
        verify(connector).initialize(any(SinkConnectorContext.class));
    }
    if (clean) {
        verify(connector).start(CONFIG);
        verify(listener).onStartup(CONNECTOR);
    }
}

(clean isn't the most accurate term here since there are clean startups in the paused state that would involve passing false for that parameter; feel free to replace with something better)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one for shutdown makes sense but I think with startup there are three different cases -

i) No call to connector.start or listener.onStartup (failure in initialization OR started in the paused state)
ii) Calls to both connector.start and listener.onStartup (successful start)
iii) Call to only connector.start (connector started in the paused state and then resumed OR failure on startup)

I think it might be more readable to keep these verify calls in the test methods as is rather than trying to force fit them into (in)appropriately named methods, WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's fair. LGTM 👍

Copy link
Contributor

@C0urante C0urante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Yash!

@C0urante C0urante merged commit 465d8fa into apache:trunk Aug 9, 2022
ijuma added a commit to confluentinc/kafka that referenced this pull request Aug 10, 2022
…(10 August 2022)

Trivial conflict in gradle/dependencies.gradle due to the newer Netty
version in confluentinc/kafka.

* apache-github/trunk:
MINOR: Upgrade gradle to 7.5.1 and bump other build/test dependencies
(apache#12495)
KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is
not eligible to join ISR in ZK mode (apache#12487)
  KAFKA-14114: Add Metadata Error Related Metrics
MINOR: BrokerMetadataSnapshotter must avoid exceeding batch size
(apache#12486)
  MINOR: Upgrade mockito test dependencies (apache#12460)
KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing
partition epoch (apache#12489)
KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest
(apache#12472)
  MINOR: Update scala version in bin scripts to 2.13.8 (apache#12477)
KAFKA-14104; Add CRC validation when iterating over Metadata Log
Records (apache#12457)
  MINOR: add :server-common test dependency to :storage (apache#12488)
  KAFKA-14107: Upgrade Jetty version for CVE fixes (apache#12440)
  KAFKA-14124: improve quorum controller fault handling (apache#12447)
ijuma added a commit to franz1981/kafka that referenced this pull request Aug 12, 2022
* apache-github/trunk: (447 commits)
  KAFKA-13959: Controller should unfence Broker with busy metadata log (apache#12274)
  KAFKA-10199: Expose read only task from state updater (apache#12497)
  KAFKA-14154; Return NOT_CONTROLLER from AlterPartition if leader is ahead of controller (apache#12506)
  KAFKA-13986; Brokers should include node.id in fetches to metadata quorum (apache#12498)
  KAFKA-14163; Retry compilation after zinc compile cache error (apache#12507)
  Remove duplicate common.message.* from clients:test jar file (apache#12407)
  KAFKA-13060: Replace EasyMock and PowerMock with Mockito in WorkerGroupMemberTest.java (apache#12484)
  Fix the rate window size calculation for edge cases (apache#12184)
  MINOR: Upgrade gradle to 7.5.1 and bump other build/test dependencies (apache#12495)
  KAFKA-14140: Ensure an offline or in-controlled-shutdown replica is not eligible to join ISR in ZK mode (apache#12487)
  KAFKA-14114: Add Metadata Error Related Metrics
  MINOR: BrokerMetadataSnapshotter must avoid exceeding batch size (apache#12486)
  MINOR: Upgrade mockito test dependencies (apache#12460)
  KAFKA-14144:; Compare AlterPartition LeaderAndIsr before fencing partition epoch (apache#12489)
  KAFKA-14134: Replace EasyMock with Mockito for WorkerConnectorTest (apache#12472)
  MINOR: Update scala version in bin scripts to 2.13.8 (apache#12477)
  KAFKA-14104; Add CRC validation when iterating over Metadata Log Records (apache#12457)
  MINOR: add :server-common test dependency to :storage (apache#12488)
  KAFKA-14107: Upgrade Jetty version for CVE fixes (apache#12440)
  KAFKA-14124: improve quorum controller fault handling (apache#12447)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants