Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test generation timeout is exceeded 15 sec - 1 min for every run #1028

Closed
Tracked by #1187
alisevych opened this issue Sep 28, 2022 · 3 comments
Closed
Tracked by #1187

Test generation timeout is exceeded 15 sec - 1 min for every run #1028

alisevych opened this issue Sep 28, 2022 · 3 comments
Assignees
Labels
ctg-bug Issue is a bug status-verified Bug fix is verified

Comments

@alisevych
Copy link
Member

alisevych commented Sep 28, 2022

Description

Test generation timeout is exceeded 15 sec approximately for every UTBot run.

To Reproduce

  1. Run the 'UTBotJava' project in IntelliJ Idea 2022.1.4
  2. Install one of the latest from main branch
  3. Open test generation dialog for utbot-sample/src/main/java/org/utbot/examples/algorithms/ArraysQuickSort.java for example
  4. Set test generation timeout to 20 sec.
  5. Check when test generation finishes.

Expected behavior

Tests are supposed to be generated during the defined timeout of 20 sec.

Actual behavior

Tests are generation for about 30-35 sec. (on the exact PC)

Environment

Windows 10 Pro
RAM 32.0 GB
Processor Intel(R) Core(TM) i7-9700K CPU @ 3.60GHz 3.60 GHz
IntelliJ IDEA 2022.1.4
Heap memory allocated for IDEA is 2 GB

Additional context

If test generation timeout is set to 60 sec, test generation continues for 75 sec approx.

As discussed with @sergeypospelov and @Markoutte :
Summaries generation and Code generation is not taken into account.
Though currently some of them seems to take much time.
Investigation is needed.

@alisevych alisevych added the ctg-bug Issue is a bug label Sep 28, 2022
@korifey korifey moved this to Todo in UTBot Java Sep 28, 2022
@alisevych alisevych added the status-help-wanted Extra attention is needed label Sep 28, 2022
@alisevych alisevych added this to the 2022.10.1 Release milestone Oct 7, 2022
@alisevych
Copy link
Member Author

alisevych commented Oct 10, 2022

Reproducing on one of the latest builds from main - unit-test-bot/rc10102022

ArraysQuickSort

Test generation timeout was set to 20 sec. Progress-bar was running up to the end during 20 sec.
Then after 15-17 seconds the test class was created.

ArrayCastExample

Test generation timeout was set to 60 sec. Progress-bar was running up to the end during 20 sec.
Then after 15 seconds the test class was created.

Heap

Test generation timeout was set to 20 sec. Progress-bar was running up to the end during 20 sec.
Then after 50-55 seconds more the test class is created. Checked twice.

@alisevych
Copy link
Member Author

For testability:
it would be great to add info log records for different phases of UnitTestBot test generation.

  • Test generation initiated
  • Collecting information (classes)
  • Fuzzing phase
  • Symbolic analysis phase
  • Minimization
  • Summaries
  • Code generation

@alisevych alisevych changed the title Test generation timeout is exceeded 15 sec. approx. for every run Test generation timeout is exceeded 15 sec - 1 min for every run Oct 10, 2022
@alisevych alisevych added status-verified Bug fix is verified and removed status-help-wanted Extra attention is needed labels Oct 17, 2022
@alisevych
Copy link
Member Author

The issue was fixed by #1151
Verified it on build from the latest main.

Repository owner moved this from Todo to Done in UTBot Java Oct 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ctg-bug Issue is a bug status-verified Bug fix is verified
Projects
Archived in project
Development

No branches or pull requests

3 participants