Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate test by fuzzing for methods with no parameters #511 #515

Merged
merged 1 commit into from
Jul 14, 2022

Conversation

Markoutte
Copy link
Collaborator

Description

This PR resolves several issues:

  • This instance now created with any public constructor. Reflection is used only if there's no another ways to create this instance
  • ModelProvider changed its signature to provide a sequence. This is needed to supply values lazily (for example, generate first 10 values and take random)
  • For methods without parameters fuzzer will generate models to change this instance
  • No more fallback for a regular value generating. Thus, fuzzer doesn't generate test using reflection anymore (except case with this instance)

Fixes #511

Type of Change

  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Automated Testing

org.utbot.framework.plugin.api.ModelProviderTest

Manual Scenario

Test method foo from this example:

package com.company.objects;

public class TestThisInstance {

    MyObject value;

    TestThisInstance() {}

    TestThisInstance(int value) {
        this.value = new MyObject();
        this.value.setField(value);
    }

    public boolean foo() {
        if (value.getField() < 100 || value.getField() == 102) {
            return true;
        }
        return value.getField() % 2 == 0;
    }

}

class MyObject {
    private int field;

    public int getField() {
        return field;
    }

    public void setField(int field) {
        this.field = field;
    }
}

Checklist (remove irrelevant options):

  • The change followed the style guidelines of the UTBot project
  • Self-review of the code is passed
  • The change contains enough commentaries, particularly in hard-to-understand areas
  • New documentation is provided or existed one is altered
  • No new warnings
  • All tests pass locally with my changes

@Markoutte Markoutte added the comp-fuzzing Issue is related to the fuzzing label Jul 13, 2022
@Markoutte Markoutte requested a review from dtim July 13, 2022 08:58
Copy link
Collaborator

@dtim dtim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of minor comments, but generally looks OK.

fallbackModelProvider.toModel(methodUnderTest.clazz).apply {
ObjectModelProvider { nextDefaultModelId++ }.withFallback(fallbackModelProvider).generate(
FuzzedMethodDescription("thisInstance", voidClassId, listOf(methodUnderTest.clazz.id), constantValues)
).take(10).shuffled(Random(0)).map { it.value.model }.first().apply {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you purposely using the fixed seed here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, fuzzer should randomize but be reproducable for same method

* @param index of the parameter in method signature
* @param value fuzzed values
*/
class FuzzedParameter(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this class be a data class?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO, data classes creates hashCode/equals/toString that aren't necessary in most cases

@@ -462,7 +486,7 @@ class UtBotSymbolicEngine(
fullPath = emptyList(),
coverage = concreteExecutionResult.coverage,
testMethodName = testMethodName?.testName,
displayName = testMethodName?.displayName
displayName = testMethodName?.takeIf { hasMethodUnderTestParametersToFuzz }?.displayName
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this code right, we will not have display name for tests without parameters at all to avoid explicit "this" argument in the name?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's true

@Markoutte Markoutte merged commit d3b10e2 into main Jul 14, 2022
@Markoutte Markoutte deleted the pelevin/this_instance_fuzzing branch July 14, 2022 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Generate test by fuzzing for methods with no parameters
2 participants