Skip to content

Fuzzer doesn't generate anything when given many recursive seeds #2434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
IlyaMuravjov opened this issue Jul 21, 2023 · 0 comments · Fixed by #2435
Closed

Fuzzer doesn't generate anything when given many recursive seeds #2434

IlyaMuravjov opened this issue Jul 21, 2023 · 0 comments · Fixed by #2435
Assignees
Labels
comp-fuzzing Issue is related to the fuzzing ctg-bug Issue is a bug

Comments

@IlyaMuravjov
Copy link
Collaborator

Description

Fuzzer doesn't generate any values when given 100 recursive seeds, even though Configuration.recursionTreeDepth is kept at default value (i.e. 4).

Note: fuzzer doesn't need to explore all $100^4$ combinations to produce something, it can explore any single one of them and that will already be enough to produce one value.

Real world example

When using static methods for creating values is supported, there will be 50 seeds for generating java.time.LocalDateTime type, most of which require another instance of java.time.LocalDateTime or some other type that also has a lot of static methods that can create it.

To Reproduce

Run the following unit test.

@Test
@Timeout(10, unit = TimeUnit.SECONDS) // withTimeout(1000) works inconsistently
fun `fuzzer works when there are many recursive seeds`() {
    class Node(val parent: Node?)
    runBlocking {
        var seenAnything = false
        withTimeout(1000) {
            runFuzzing(
                { _, _ -> List(100) {Seed.Recursive<Unit, Node?>(
                    construct = Routine.Create(listOf(Unit)) { (parent) -> Node(parent) },
                    modify = emptySequence(),
                    empty = Routine.Empty { null }
                )}.asSequence() },
                Description(listOf(Unit))
            ) { _, _ ->
                seenAnything = true
                BaseFeedback(Unit, Control.STOP)
            }
        }
        assertTrue(seenAnything) { "Fuzzer hasn't generated any values" }
    }
}

Expected behavior
Test passes.

Actual behavior
Test fails.

Visual proofs (screenshots, logs, images)

fuzzer works when there are many recursive seeds() timed out after 10 seconds
@IlyaMuravjov IlyaMuravjov added ctg-bug Issue is a bug comp-fuzzing Issue is related to the fuzzing labels Jul 21, 2023
@IlyaMuravjov IlyaMuravjov self-assigned this Jul 21, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in UTBot Java Jul 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing ctg-bug Issue is a bug
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant