Skip to content

Missing errors in SARIF report and on Problems view #2023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alisevych opened this issue Mar 23, 2023 · 0 comments
Open

Missing errors in SARIF report and on Problems view #2023

alisevych opened this issue Mar 23, 2023 · 0 comments
Assignees
Labels
comp-sarif Issue is related to SARIF report or Problems tab ctg-bug Issue is a bug

Comments

@alisevych
Copy link
Member

Description

Some test methods from Error suites are missing on Problems view and in SARIF report.
Probably the issue scope is bigger.

To Reproduce

  1. Run the 'utbot' project in IntelliJ Idea 2022.3 Ultimate
  2. Install plugin built from unit-test-bot/rc2023.3 branch
  3. Use plugin to Generate and Run tests for types/Generics with default settings
  4. Wait Problems view is opened and tests are run

Expected behavior

All test methods from error suites should be shown on Problems view.

Actual behavior

9 tests are failing, 7 are displayed on Problems view.
2 tests are missing in SARIF report either.

Visual proofs (screenshots, logs, images)

Comparing Error suites and Problems view, these 2 are missing:

    @Test
    @DisplayName("mapAsNonStaticField: object.nonStaticMap.put(\"key\", \"value\") : True -> ThrowNullPointerException")
    public void testMapAsNonStaticField_MapPut() {
        Generics generics = new Generics();
        CollectionAsField object = new CollectionAsField();
        object.nonStaticMap = null;
        
        /* This test fails because method [org.utbot.examples.types.Generics.mapAsNonStaticField] produces [java.lang.NullPointerException]
            org.utbot.examples.types.Generics.mapAsNonStaticField(Generics.java:27) */
        generics.mapAsNonStaticField(object);
    }

    @Test
    @DisplayName("methodWithArrayTypeBoundary:  -> throw NullPointerException")
    public void testMethodWithArrayTypeBoundaryThrowsNPE() {
        Generics generics = new Generics();
        
        /* This test fails because method [org.utbot.examples.types.Generics.methodWithArrayTypeBoundary] produces [java.lang.NullPointerException]
            org.utbot.examples.types.ArrayTypeParameters.methodWithArrayTypeBoundary(Generics.java:51)
            org.utbot.examples.types.Generics.methodWithArrayTypeBoundary(Generics.java:45) */
        generics.methodWithArrayTypeBoundary();
    }

image

image

Environment

Windows 10 Pro
IntelliJ IDEA 2022.3.3 Ultimate
Gradle, JDK 17

Additional context

Checked in a new Gradle project with JDK 8 - 1 test was missing

@alisevych alisevych added the ctg-bug Issue is a bug label Mar 23, 2023
@alisevych alisevych added the comp-sarif Issue is related to SARIF report or Problems tab label Mar 23, 2023
@alisevych alisevych modified the milestone: 2023.03 Release Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-sarif Issue is related to SARIF report or Problems tab ctg-bug Issue is a bug
Projects
Status: Todo
Development

No branches or pull requests

2 participants