Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WFCORE-6214] Fixed Flaky Test HelpSupportTestCase.testStandalone #5365

Closed
wants to merge 1 commit into from

Conversation

RishabhPrabhu5
Copy link
Contributor

@RishabhPrabhu5 RishabhPrabhu5 commented Jan 30, 2023

What is the purpose of this PR

  • This PR fixes the error resulting from the flaky test org.jboss.as.cli.impl.aesh.HelpSupportTestCase
  • The mentioned test may fail or pass without changes made to the source code when it is run in different JVMs due to HashSet's non-deterministic iteration order.

Why the test fails

  • HelpSupportTestCase calls SynopsisGenerator, which uses a HashSet as its data structure for the conflicts object. As per Java 11 documentation, the ordering of HashSet is not constant, so assigning the same HashSet as an array to different objects can change the ordering. So, when comparing these two arrays with assertEquals the test may fail as the given order can be different from the expected order.

Reproduce the test failure

  • Run the test with NonDex maven plugin. The command to recreate the flaky test failure is
    mvn -pl cli edu.illinois:nondex-maven-plugin:2.1.1:nondex - Dtest=org.jboss.as.cli.impl.aesh.HelpSupportTestCase#testStandalone
  • Fixing the flaky test now may prevent flaky test failures in future Java versions.

Expected result

  • The tests should run successfully when run with NonDex.
  • Expected: <...server-groups] | [--[replace] | [--server-groups]] )>

Actual Result

  • We get the failure:
  • [ERROR] testStandalone(org.jboss.as.cli.impl.aesh.HelpSupportTestCase) Time elapsed: 0.254 s <<< FAILURE!
    org.junit.ComparisonFailure: org.jboss.as.cli.impl.aesh.Commands$Standalone$Command10. EXPECTED [command1 [<argument>] ( [--all-server-groups] | [--replace] | [--server-groups] )]. FOUND [command1 [<argument>] ( [--all-server-groups] | [--server-groups] | [--replace] )] expected: <...server-groups] | [--[replace] | [--server-groups]] )> but was: <...server-groups] | [--[server-groups] | [--replace]] )>

Fix

  • Changed the conflicts and conflicts2 objects from HashSet to LinkedHashSet in method SynopsisGenerator.addSynopsisOption(SynopsisOption) and imported LinkedHashSet.

This PR is similar to some merged PRs in other wildfly projects:

wildfly/wildfly#13728
wildfly/wildfly#11833
wildfly/jboss-ejb-client#534

Jira Issue:

https://issues.redhat.com/browse/WFCORE-6214

@RishabhPrabhu5 RishabhPrabhu5 changed the title Fixed Flaky Test HelpSupportTestCase.testStandalone [WFCORE-6214] Fixed Flaky Test HelpSupportTestCase.testStandalone Jan 30, 2023
@wildfly-ci
Copy link

Hello, RishabhPrabhu5. I'm waiting for one of the admins to verify this patch with /ok-to-test in a comment.

@RishabhPrabhu5 RishabhPrabhu5 deleted the fix-merge branch January 30, 2023 03:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants