Skip to content

UTBot Python cannot find Division by Zero problem #1831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tyuldashev opened this issue Feb 20, 2023 · 2 comments · Fixed by #1893
Closed

UTBot Python cannot find Division by Zero problem #1831

tyuldashev opened this issue Feb 20, 2023 · 2 comments · Fixed by #1893
Assignees
Labels
comp-codegen Issue is related to code generator ctg-enhancement New feature, improvement or change request lang-python Issue is related to python support status-verified Bug fix is verified

Comments

@tyuldashev
Copy link
Collaborator

Description

Even when all types are specified UTBot Python doesn't find case when Division by Zero happens. Both UTBotJava and UTBotCPP can find such problem and it seems quite classic example of the problem which could be revealed with static analysis, so I would expect that UTBot Python also find the error.

To Reproduce

Steps to reproduce the behavior:

def div(a: int, b: int) -> int:
    return a / b
  1. Invoke code generation on code above

Expected behavior

Test which exposes 'Division by Zero' problem is generated among the others.

Actual behavior

No such test is generated, the only test looks like that:

    def test_div(self):
        """
        a = 4
        b = 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111
        """
        actual = basics.div(4, 111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111)
        
        self.assertEqual(0.0, actual)
@tyuldashev tyuldashev added ctg-enhancement New feature, improvement or change request comp-codegen Issue is related to code generator lang-python Issue is related to python support labels Feb 20, 2023
@tyuldashev
Copy link
Collaborator Author

Suggestion is to allow UTBot to proceed test generation/execution till timeout with to hope to find possible errors (exceptions) in the code as it's done in Java UTbot. For sure this can be done only after equal time distribution is implemented #1820.

@denis-fokin
Copy link
Collaborator

We should distinguish cases with different results. From the point of our implementation we covered the path, but arithmetic exception and regular exit are different outcome and we must consider them.

Additionally, we can try to work in three modes depending on the task:

  1. Try to cover all paths and exit
  2. If we find an error, let's try to find the non-error execution
  3. We covered everything with non-error executions but still have time, let's spend the time with benefits and try to find possible errors

@tyuldashev tyuldashev added this to the 2023.03 Release milestone Mar 1, 2023
@github-project-automation github-project-automation bot moved this from Todo to Done in UTBot Java Mar 9, 2023
@tyuldashev tyuldashev added the status-verified Bug fix is verified label Mar 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-codegen Issue is related to code generator ctg-enhancement New feature, improvement or change request lang-python Issue is related to python support status-verified Bug fix is verified
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

3 participants