Module-private JDK packages are used in generated tests

[dtim]

Description

Starting from version 9, JDK is modularized. Each module explicitly lists packages that it makes available to other (possibly specific) modules, all other packages are by default module-private. It is impossible to use private packages at compile time (e.g., we can't import them), and special actions are necessary to access them at runtime using reflection (see A Guide to Java 9 Modularity for high-level survey).

If a JDK method internally uses classes from a module-private package, UnitTestBot can produce test cases that import these packages. This leads to compilation errors (e.g., if the test case tries to explicitly import something like jdk.internal.misc.Unsafe).

Accessing these classes via reflection also requires special care and should be avoided when possible.

To Reproduce

Create a project using JDK 11 for compilation.

Generate a test suite for the following class:

public class IntArrayBasics {
    public int arrayEqualsExample(int[] arr) {
        boolean a = Arrays.equals(arr, new int[]{1, 2, 3});
        if (a) {
            return 1;
        } else {
            return 2;
        }
    }
}

Expected behavior

A nice test suite that compiles and covers actual execution paths.

Actual behavior

Generated test suite imports jdk.internal.misc.Unsafe, gets the corresponding static field (Unsafe.theUnsafe), reassigns it, and then sets some static fields of Unsafe itself. The resulting code does not compile, test cases check for obscure exceptions, and the test suite is generally weird.

Here is a fragment of the generated code.

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.DisplayName;
import jdk.internal.misc.Unsafe;

import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
import static org.utbot.runtime.utils.UtUtils.getStaticFieldValue;
import static org.utbot.runtime.utils.UtUtils.createInstance;
import static org.utbot.runtime.utils.UtUtils.setStaticField;

public class IntArrayBasicsTest {
    /**
     * <pre>
     * Test
     * throws Error in: boolean a = Arrays.equals(arr, new int[] { 1, 2, 3 });
     * </pre>
     */
    @Test
    @DisplayName("arrayEqualsExample: a = Arrays.equals(arr, new int[] { 1, 2, 3 }) : True -> ThrowError")
    public void testArrayEqualsExample_ThrowError_2() throws Exception {
        Class unsafeClazz = Class.forName("jdk.internal.misc.Unsafe");
        Unsafe prevTheUnsafe = ((Unsafe) getStaticFieldValue(unsafeClazz, "theUnsafe"));
        boolean prevBE = ((Boolean) getStaticFieldValue(unsafeClazz, "BE"));
        int prevARRAY_INT_BASE_OFFSET = Unsafe.ARRAY_INT_BASE_OFFSET;
        try {
            Unsafe theUnsafe = ((Unsafe) createInstance("jdk.internal.misc.Unsafe"));
            setStaticField(unsafeClazz, "theUnsafe", theUnsafe);
            setStaticField(unsafeClazz, "BE", false);
            setStaticField(unsafeClazz, "ARRAY_INT_BASE_OFFSET", 0);
            IntArrayBasics intArrayBasics = new IntArrayBasics();
            int[] arr = {1, -255, -255};

            assertThrows(Error.class, () -> intArrayBasics.arrayEqualsExample(arr));
        } finally {
            setStaticField(Unsafe.class, "theUnsafe", prevTheUnsafe);
            setStaticField(Unsafe.class, "BE", prevBE);
            setStaticField(Unsafe.class, "ARRAY_INT_BASE_OFFSET", prevARRAY_INT_BASE_OFFSET);
        }
    }
}

Environment

The plugin should be run on JDK 11, the project should use JDK 11 as well.

Additional context

The general problem can be illustrated by the specific problem with Arrays.equals() call described above. In JDK 11, Arrays#equals calls jdk.internal.util.ArraysSupport#mismatch method, and its body contains the static call:

i = vectorizedMismatch(
                    a, Unsafe.ARRAY_INT_BASE_OFFSET,
                    b, Unsafe.ARRAY_INT_BASE_OFFSET,
                    length, LOG2_ARRAY_INT_INDEX_SCALE);

Here we can see references to jdk.internal.misc.Unsafe, and the general logic of execution state initialization leads the engine to accessing the static Unsafe instance and its static fields.

The resulting code is incorrect in several ways: it does not compile, it is unreadable, and produced exception-throwing executions are false positive: we get exceptions only because we are trying to do incorrect operations on statics, not because the method under test is buggy.

The simple fix of adding jdk package group into the list of trusted libraries (see commit d61f887) is not sufficient. We get rid of bad import statement, and the code compiles, but resulting tests fail (exceptions are expected, but they are not actually thrown).

Related issue: #636 (we can access platform-specific classes that are not available on other platforms with the same JDK release). The conceptual root of the problem is the same: we are going "too deep" into JDK code, and the resulting execution becomes hard to reproduce in a new environment.