Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fixed Flaky Tests Caused by JSON permutations
###Description Flaky Tests found using NonDex by running the commands - mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentComplicatedJsonObject mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentSimpleJsonArray mvn -pl . edu.illinois:nondex-maven-plugin:2.1.1:nondex -Dtest=org.json.junit.XMLTest#testIndentSimpleJsonObject The logged failure was- [ERROR] Failures: [ERROR] XMLTest.testIndentSimpleJsonObject:1193 expected:<...> <employee> <[married>true</married> <name>sonoo</name> <salary>56000</salary]> </employee> </Te...> but was:<...> <employee> <[name>sonoo</name> <salary>56000</salary> <married>true</married]> </employee> </Te...> The issue is the same for all three tests, so here I only show the failure message for the third test (to reduce the length of the error message). ### Investigation The tests fail with a comparison error while comparing an expected JSON String and the result from the value returned from XML.toString(). The toString function of XML makes no guarantees as to the iteration order of the attributes in the object. This makes the test outcome non-deterministic, and the test fails whenever the function returns a mismatch in order of the elements in the JSON String. To fix this, the expected and actual keys should be checked in a more deterministic way so that the assertions do not fail. ### Fix Expected and Actual values can be converted into JSONObject and the similar function can be used to compare these objects. As this function compares the values inside the JSONObjects without needing order, the test becomes deterministic and ensures that the flakiness from the test is removed. The PR does not introduce a breaking change.
- Loading branch information