tacas

The link to the permanent online repository is https://zenodo.org/uploads/10124958. An artifact can be downloaded here as a .ZIP archive containing UnitTestBot source code.

It can be also downloaded via a direct link: https://github.com/UnitTestBot/UTBotJava/releases/download/2023.10/utbot-intellij-2023.10.zip

How to install and use UnitTestBot?

UnitTestBot is a plugin for IntelliJ IDEA.

To reproduce test generation results, you need to install UnitTestBot (the artifact provided) into your IntelliJ IDEA ("File" > "Settings" > "Plugins" > "Install plugin from disk").

For each test generation, one should specify the following:

• Test generation method (symbolic execution or fuzzing or both). Go to "File" > "Settings" > "Tools" > "UnitTestBot" and set the proportion with a slider.
• Please disable the "Enable summaries generation" checkbox, but this step is not crucial.

To generate tests for a method or a class:

1. Open a method or a class in an IDE editor.
2. Locate the cursor inside this method or class.
3. Press Shift + Alt + U or right-click and choose to "Generate Tests with UnitTestBot".
4. Change settings in the dialog window if required.
5. Press OK — he class with generated tests will be opened as soon as the generation is completed.

Please specify the most important settings:

• "Tests type": unit or integration tests
• "Spring configuration": no configuration or one of the existing ones

To generate tests for the whole package

1. Right-click on it in the Project tool window in the IDE.
2. Choose to "Generate Tests with UnitTestBot".
3. Further steps are similar to those for generating tests for a method or a class.

How to reproduce “small” examples?

All the "small" examples mentioned in the article are located in the following repository (as cited in the paper): https://github.com/UnitTestBot/UnitTestBot-Spring-demo/tree/fd9c802d5240402bd3a67e1aabad6ab553cd215b

The methods under tests are listed in the README section. The first column is the name of the method under test, the second one describes generation settings while the third one shows the name of the class with the tests generated.

The most representative examples are:

For Section 3.2: Standard unit tests — OrderService.isMajorityExpensive For Section 3.3: Spring-specific unit tests — CalculatorService.calcSquareSum For Section 3.4: Integration tests — OrderService.isNotEmpty

Generated tests are located in classes (respectively):

• OrderServiceUnitTests,
• CalculatorServiceWithConfigurationTests,
• OrderServiceIntegrationTests.

It may also be interesting to pay attention to the example of the generated tests for the controllers located in OrderControllerTests.

To reproduce test generation, please use the following settings:

• OrderService.isMajorityExpensive 100% symbolic, Unit tests, No configuration

• CalculatorService.calcSquareSum 100% symbolic, Unit tests, Spring configuration = DemoApplication

• OrderService.isNotEmpty 100% fuzzing, Integration tests, Spring configuration = DemoApplication

• OrderController.isMajorityExpensive 100% fuzzing, Integration tests, Spring configuration = DemoApplication

Please note that there are other possible proportions of time allocated for symbolic execution or fuzzing that allow us to get the same generated tests. The suggested proportions of 0/100 or 100/0 are used solely for better reproducibility.

In the listing from the section 3.3, classes and methods are named differently than in this demo repository, but structurally there is no difference. So all the mentioned instructions to generate tests for CalculatorService.calcSquareSum can be used also for SpeciesService.getSpecies from the paper.

How to reproduce “coverage” experiments?

1. Install UnitTestBot as an IntelliJ IDEA plugin.
2. Download the mentioned projects from GitHub: https://github.com/jirkapinkas/java-blog-aggregator-boot/tree/f83382481036c26e700df1e5e90ec3d1b7d433a3 https://github.com/MathAndMedLab/Medical-Web-App/tree/9cff06c080a03d24c7326788aea42fcad9015c12
3. For Medical-Web-App, complete the local deployment instruction from the repository README before generating integration tests.
4. Set the “Test generation method” proportion to 70/30.
5. Generate tests for the main package of the project source root.

Please use one of the following configurations. a) Use the default options in the dialog window and press “OK”. Standard unit tests will be generated. b) Use Application and MedicalWebApp classes for the first and the second project respectively as configuration classes. Spring-specific unit tests will be generated.

6. Remove uncompilable tests.
7. Find test source root with the generated tests in the project. Right-click on it and choose More run/Debug > Run tests with coverage.

To reproduce the experiments for controllers, generate tests for the classes annotated with @Controller or @RestController annotations — from these packages:

• com.app.medicalwebapp.controllers
• cz.jiripinkas.jba.controller
• cz.jiripinkas.jba.controller.admin

Use Application and MedicalWebApp classes as configuration classes and choose integration tests generation as opposed to previously used unit tests.

How to reproduce the “bug-finding” ability experiments?

1. Install UnitTestBot as IntelliJ IDEA plugin.
2. Download the project from GitHub: https://github.com/SalithaUCSC/spring-boot-testing/tree/152e5cbbe85d0e59bc28aad87aaca47ed2841641
3. Set the “Test generation method” proportion to 70/30.
4. For the Order.equals method, generate unit tests with Spring configuration = NoConfiguration.
5. For the OrderController.deleteOrderById method, generate integration tests with Spring configuration = OrderServiceApplication.

Some of the failing generated tests will be similar to those described in issues: https://github.com/SalithaUCSC/spring-boot-testing/issues/6 https://github.com/SalithaUCSC/spring-boot-testing/issues/7