Updated Generator.java to fix the flaky errors #727
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
PR Overview:
This PR fixes the flaky/non-deterministic behavior of the following test because it assumes the ordering of methods returned by
getDeclaredMethods()
org.bytedeco.javacpp.BufferTest
org.bytedeco.javacpp.EnumTest
org.bytedeco.javacpp.PointerTest
Test Overview:
In all of the above tests, the tests call the
setUpClass()
method where the problem arises. ThesetUpClass()
makes an internal to theGenerator
class’s object where the problem resides.You can reproduce the issue by running the following commands (the below command is for BufferTest, please change class name for targeting other tests) -
Root Cause
Method[] methods = cls.getDeclaredMethods();
In the
Generator
class, thecls.getDeclaredMethods()
method returns an array with values in a non-deterministic order, leading to the core of the issue. Within thefunctionMethods()
method of theGenerator
class, there's a call tocls.getDeclaredMethods()
used to populate thecallbackAllocators[]
array. Additionally, the samecls.getDeclaredMethods()
method is invoked in themethods()
method of theGenerator
class, which utilizes thecallbackAllocators
array. However, due to the differing order of results produced bycls.getDeclaredMethods()
, it results in a test failure.This flakiness was identified by the nondex tool created by the researchers of UIUC
Fix:
To fix the issue I decided to make the output of
callbackAllocators
array deterministic by ordering thecls.getDeclaredMethods()
result both inmethods()
method andfunctionMethods()
method.https://github.com/njain2208/javacpp/blob/4c62303548be1b9edc3e22d23d3674c397ff9829/src/main/java/org/bytedeco/javacpp/tools/Generator.java#L2085-L2091
https://github.com/njain2208/javacpp/blob/4c62303548be1b9edc3e22d23d3674c397ff9829/src/main/java/org/bytedeco/javacpp/tools/Generator.java#L3643-L3649