Skip to content

K14130: Reduce RackAwarenesssTest to unit Test#12476

Merged
guozhangwang merged 8 commits intoapache:trunkfrom
guozhangwang:K1XXXX-reduce-rackawarenesss-to-ut
Aug 3, 2022
Merged

K14130: Reduce RackAwarenesssTest to unit Test#12476
guozhangwang merged 8 commits intoapache:trunkfrom
guozhangwang:K1XXXX-reduce-rackawarenesss-to-ut

Conversation

@guozhangwang
Copy link
Contributor

While working on KAFKA-13877, I feel it's an overkill to introduce the whole test class as an integration test, since all we need is to just test the assignor itself which could be a unit test. Running this suite with 9+ instances takes long time and is still vulnerable to all kinds of timing based flakiness. A better choice is to reduce it as a unit test, similar to HighAvailabilityStreamsPartitionAssignorTest that just test the behavior of the assignor itself, rather than creating many instances hence depend on various timing bombs to not explode.

Since we mock everything, there's no flakiness anymore. Plus we greatly reduced the test runtime (on my local machine, the old integration takes about 35 secs to run the whole suite, while the new one take 20ms on average).

Committer Checklist (excluded from commit message)

  • Verify design and implementation
  • Verify test coverage and CI build status
  • Verify documentation (including upgrade notes)

@guozhangwang
Copy link
Contributor Author

ping @vvcephei for a quick review.

import static org.apache.kafka.streams.processor.internals.assignment.AssignmentTestUtils.UUID_8;
import static org.apache.kafka.streams.processor.internals.assignment.AssignmentTestUtils.UUID_9;
import static org.apache.kafka.streams.processor.internals.assignment.StreamsAssignmentProtocolVersions.LATEST_SUPPORTED_VERSION;
import static org.easymock.EasyMock.anyObject;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are actively trying to get rid of EasyMock/PowerMock from all classes. Please use Mockito in this new code.

Here are a couple of helpful tips that you might useful:

  1. EasyMock.createNiceMock/EasyMock.createMock can be replaced with mock from Mockito. All mocks in Mockito are nice.
  2. Mockito doesn't require replay so you can get rid of those lines completely.
  3. You might want to consider running the test using @RunWith(MockitoJUnitRunner.StrictStubs.class) since it comes with a lot of benefits highlighted in the docs https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/junit/MockitoJUnitRunner.html

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @divijvaidya , I've updated the class to switch to mockito.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[optional]

@guozhangwang, if you add @RunWith(MockitoJUnitRunner.StrictStubs.class) at class level and re-run all tests you will see following exception (or warning, it depends on how you look at it):

Unnecessary stubbings detected in test class: RackAwarenessStreamsPartitionAssignorTest
Clean & maintainable test code requires zero unnecessary code.
Following stubbings are unnecessary (click to navigate to relevant line of code):
  1. -> at org.apache.kafka.streams.processor.internals.RackAwarenessStreamsPartitionAssignorTest.createMockTaskManager(RackAwarenessStreamsPartitionAssignorTest.java:173)
Please remove unnecessary stubbings or use 'lenient' strictness. More info: javadoc for UnnecessaryStubbingException class.

To read more about MockitoJunitRunner please check the official docs at: https://javadoc.io/doc/org.mockito/mockito-core/latest/org/mockito/junit/MockitoJUnitRunner.html

@guozhangwang
Copy link
Contributor Author

I'm merging in this PR now since it includes a hotfix on [StreamTaskTest.java] to turn the build to green.

@guozhangwang guozhangwang merged commit 5ceaa58 into apache:trunk Aug 3, 2022
ijuma added a commit to confluentinc/kafka that referenced this pull request Aug 5, 2022
…(5 August 2022)

Version related conflicts:
* Jenkinsfile
* gradle.properties
* streams/quickstart/java/pom.xml
* streams/quickstart/java/src/main/resources/archetype-resources/pom.xml
* streams/quickstart/pom.xml
* tests/kafkatest/__init__.py
* tests/kafkatest/version.py

* commit 'add7cd85baa61cd0e1430': (66 commits)
KAFKA-14136 Generate ConfigRecord for brokers even if the value is
unchanged (apache#12483)
  HOTFIX / KAFKA-14130: Reduce RackAwarenesssTest to unit Test (apache#12476)
  MINOR: Remove ARM/PowerPC builds from Jenkinsfile (apache#12380)
  KAFKA-14111 Fix sensitive dynamic broker configs in KRaft (apache#12455)
  KAFKA-13877: Fix flakiness in RackAwarenessIntegrationTest (apache#12468)
KAFKA-14129: KRaft must check manual assignments for createTopics are
contiguous (apache#12467)
KAFKA-13546: Do not fail connector validation if default topic
creation group is explicitly specified (apache#11615)
KAFKA-14122: Fix flaky test
DynamicBrokerReconfigurationTest#testKeyStoreAlter (apache#12452)
  MINOR; Use right enum value for broker registration change (apache#12236)
  MINOR; Synchronize access to snapshots' TreeMap (apache#12464)
  MINOR; Bump trunk to 3.4.0-SNAPSHOT (apache#12463)
  MINOR: Stop logging 404s at ERROR level in Connect
KAFKA-14095: Improve handling of sync offset failures in MirrorMaker
(apache#12432)
  Minor: enable index for emit final sliding window (apache#12461)
  MINOR: convert some more junit tests to support KRaft (apache#12456)
  KAFKA-14108: Ensure both JUnit 4 and JUnit 5 tests run (apache#12441)
  MINOR: Remove code of removed metric (apache#12453)
MINOR: Update comment on verifyTaskGenerationAndOwnership method in
DistributedHerder
KAFKA-14012: Add warning to closeQuietly documentation about method
references of null objects (apache#12321)
  MINOR: Fix static mock usage in ThreadMetricsTest (apache#12454)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants