Skip to content

Commit

Permalink
Squashed commit of the following:
Browse files Browse the repository at this point in the history
commit 31fbc83
Author: Andrey Tarbeev <dinis.taranov@mail.ru>
Date:   Fri Jul 8 10:50:52 2022 +0300

    Minimize exceptions in signature (#418)

    * Minimize exceptions in test methods signatures

commit 63eb9b0
Author: Yury Kamenev <yurkam447@gmail.com>
Date:   Thu Jul 7 15:13:45 2022 +0300

    Removed always used coverage-based minimization strategy (#450)

commit fe0f7f8
Author: Yury Kamenev <yurkam447@gmail.com>
Date:   Thu Jul 7 14:19:53 2022 +0300

    Disabled NPE checks for non-public library fields by default (#353)

commit d1b51fe
Author: Yury Kamenev <yurkam447@gmail.com>
Date:   Thu Jul 7 13:44:46 2022 +0300

    Fixed NPE for processing static field as first statement in MUT (#433)

commit acc2fff
Author: Nikita Vlaev <nikitavlaev00@gmail.com>
Date:   Wed Jul 6 15:37:28 2022 +0300

    Added ForceStaticMockListener to run "configure mockito-inline" action.
    Refactored url listener in notifications.

commit 80c2328
Author: Nikita Stroganov <54814796+IdeaSeeker@users.noreply.github.com>
Date:   Thu Jul 7 11:21:13 2022 +0300

    Support -P parameters in the utbot-gradle (#377)

commit a92e289
Author: Sergey Pospelov <sergeypospelov59@gmail.com>
Date:   Tue Jul 5 10:29:45 2022 +0300

    Fix searching of modifiers field for JDK 8-17

commit 9a3acb6
Author: Amandel Pie <105506115+amandelpie@users.noreply.github.com>
Date:   Wed Jul 6 14:16:09 2022 +0300

    Disabled clustering of tests generated by Fuzzer (#431)

    * Disabled clustering of tests generated by Fuzzer

    * Refactored the multiple calls

commit c09568f
Author: Vassiliy Kudryashov <vassiliy.kudryashov@gmail.com>
Date:   Wed Jul 6 10:21:52 2022 +0300

    Existing test file without test class obstruct generation #160 (#425)

commit 2ec8fb3
Author: Denis Fokin <Denis.Fokin@gmail.com>
Date:   Tue Jul 5 16:52:42 2022 +0300

    Better naming

commit 68c66ef
Author: Vassiliy Kudryashov <vassiliy.kudryashov@gmail.com>
Date:   Tue Jul 5 19:51:55 2022 +0300

    SVG logo contains 'inaccurate' holes #330 (#411)

commit 1d48789
Author: Dmitrii Timofeev <dtim@comitative.com>
Date:   Tue Jul 5 17:08:38 2022 +0300

    Workaround: run concrete execution for invokedynamic

commit 65d95c1
Author: Vassiliy Kudryashov <vassiliy.kudryashov@gmail.com>
Date:   Tue Jul 5 19:42:55 2022 +0300

    Plugin fails with an exception if no SDK is configured for the test module #399 (#424)

commit 6f9f2d8
Author: Maksim Pelevin <maks.pelevin@gmail.com>
Date:   Tue Jul 5 17:49:58 2022 +0300

    Fix CNFE: kotlin.collections.ArrayDeque in ContestEstimator (#419)

commit 67cc7d9
Author: Denis Fokin <Denis.Fokin@gmail.com>
Date:   Tue Jul 5 11:58:40 2022 +0300

    Put concrete executor jar in resources for contest estimator

commit 63e2252
Author: Alena Lisevych <37301492+alisevych@users.noreply.github.com>
Date:   Tue Jul 5 15:48:29 2022 +0300

    Tooltips correction for #343 (#410)

    Co-authored-by: Alena Lisevych <>

commit 8b3defa
Author: Vassiliy Kudryashov <vassiliy.kudryashov@gmail.com>
Date:   Tue Jul 5 13:22:31 2022 +0300

    Write access exception on test generating in Idea project with JDK 11 #350 (#409)

commit c02883c
Author: Amandel Pie <105506115+amandelpie@users.noreply.github.com>
Date:   Tue Jul 5 11:21:21 2022 +0300

    Enables 40 tests for utbot-summary module (#389)

    * Repair testPow

    * Repaired ReturnExampleTest tests

    * Restored SummaryCycleTest.kt

    * Restored all the tests

commit 65c07f8
Author: Victoria <32179813+victoriafomina@users.noreply.github.com>
Date:   Tue Jul 5 09:51:44 2022 +0300

    Update Dockerfile_java_cli (#396)

commit 3b21112
Author: Alexey Menshutin <alex.menshutin99@gmail.com>
Date:   Mon Jul 4 21:16:53 2022 +0300

    ToString support for abstract collection using makeSymbolic #391

commit c2673df
Author: Vassiliy Kudryashov <vassiliy.kudryashov@gmail.com>
Date:   Mon Jul 4 21:56:43 2022 +0300

    IDE fatal error during tests generation #68 (#379)

commit 43693fb
Author: Nikita Vlaev <vlaev.nikita@huawei.com>
Date:   Mon Jul 4 18:59:57 2022 +0300

    Bring back test class generation (#403)

commit 1c01193
Author: Yury Kamenev <yurkam447@gmail.com>
Date:   Mon Jul 4 20:24:03 2022 +0300

    Added the wrapper for static methods of java.util.List (#400)

commit a805f31
Author: Nikita Vlaev <vlaev.nikita@huawei.com>
Date:   Fri Jul 1 20:16:28 2022 +0300

    Add title pane with unsupported jdk notification (#373)

commit 16c13fb
Author: Dmitrii Timofeev <dtim@comitative.com>
Date:   Mon Jul 4 18:35:36 2022 +0300

    Initiate concrete execution if a wrapper method is missing (#392)

    If a JVM class is overridden but a method is missing from the wrapper,
    the engine will discard the path and fall back to concrete execution
    instead of analysing the real JVM code graph.

    This approach fixes the problem with methods that have been introduced
    in newer JDKs. Now wrappers are mostly limited to Java 1.8 interfaces
    and fail to analyze methods like `String::isBlank` or `String::lines`
    when the code runs under JDK 11. Building graphs from the real JDK code
    fails because the wrapper does not have private fields that the original
    code uses.

    TODO: to allow symbolic analysis of the code, missing methods should be
    actually implemented in corresponding wrappers.
  • Loading branch information
rudolf101 committed Jul 8, 2022
1 parent 1b52bdc commit bf2e0e7
Show file tree
Hide file tree
Showing 70 changed files with 3,360 additions and 1,580 deletions.
2 changes: 1 addition & 1 deletion docker/Dockerfile_java_cli
Original file line number Diff line number Diff line change
Expand Up @@ -22,5 +22,5 @@ RUN curl -H "Authorization: Bearer ${ACCESS_TOKEN}" \
&& unzip "${JAVA_CLI_ZIP_NAME}" \
&& rm "${JAVA_CLI_ZIP_NAME}"

ENV JAVA_CLI_PATH="$(find $pwd -type f -name "utbot-cli*")"
ENV JAVA_CLI_PATH="$(find /usr/src -type f -name utbot-cli*)"
RUN ln -s "${JAVA_CLI_PATH}" $pwd/utbot-cli.jar
19 changes: 10 additions & 9 deletions docs/SpeculativeFieldNonNullability.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,14 +17,15 @@ most of generated branches would be `NPE` branches, while useful paths could be

Beyond that, in many cases the `null` value of a field can't be generated using the public API
of the class. This is particularly true for final fields, especially in system classes.
Automatically generated tests assign `null` values to fields in questions using reflection,
it is also often true for non-public fields from standard library and third-party libraries (even setters often do not
allow `null` values). Automatically generated tests assign `null` values to fields using reflection,
but these tests may be uninformative as the corresponding `NPE` branches would never occur
in the real code that limits itself to the public API.

## The solution

To discard irrelevant `NPE` branches, we can speculatively mark fields we as non-nullable even they
do not have an explicit `@NotNull` annotation. In particular, we can use this approach to final
do not have an explicit `@NotNull` annotation. In particular, we can use this approach to final and non-public
fields of system classes, as they are usually correctly initialized and are not equal `null`.

At the same time, we can't always add the "not null" hard constraint for the field: it would break
Expand All @@ -38,18 +39,18 @@ no way to check whether the address corresponds to a final field, as the corresp
of the global graph would refer to a local variable. The only place where we have the complete
information about the field is this method.

We use the following approach. If the field is final and belongs to a system class,
we mark it as a speculatively non-nullable in the memory
We use the following approach. If the field belongs to a library class (according to `soot.SootClass.isLibraryClass`)
and is final or non-public, we mark it as a speculatively non-nullable in the memory
(see `org.utbot.engine.Memory.speculativelyNotNullAddresses`). During the NPE check
we will add the `!isSpeculativelyNotNull(addr(field))` constraint
to the `NPE` branch together with the usual `addr(field) == null` constraint.

For final fields, these two conditions can't be satisfied at the same time, as we speculatively
mark final fields as non-nullable. As a result, the NPE branch would be discarded. If a field
is not final, the condition is satisfiable, so the NPE branch would stay alive.
For final/non-public fields, these two conditions can't be satisfied at the same time, as we speculatively
mark such fields as non-nullable. As a result, the NPE branch would be discarded. If a field
is public or not final, the condition is satisfiable, so the NPE branch would stay alive.

We limit this approach to the system classes only, because it is hard to speculatively assume
something about non-nullability of final fields in the user code.
We limit this approach to the library classes only, because it is hard to speculatively assume
something about non-nullability of final/non-public fields in the user code.

The same approach can be extended for other cases where we want to speculatively consider some
fields as non-nullable to prevent `NPE` branch generation.
10 changes: 9 additions & 1 deletion utbot-core/src/main/kotlin/org/utbot/common/ReflectionUtil.kt
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,15 @@ object Reflection {
unsafe = f.get(null) as Unsafe
}

private val modifiersField: Field = Field::class.java.getDeclaredField("modifiers")

// TODO: works on JDK 8-17. Doesn't work on JDK 18
private val modifiersField: Field = run {
val getDeclaredFields0 = Class::class.java.getDeclaredMethod("getDeclaredFields0", Boolean::class.java)
getDeclaredFields0.isAccessible = true
@Suppress("UNCHECKED_CAST")
val fields = getDeclaredFields0.invoke(Field::class.java, false) as Array<Field>
fields.first { it.name == "modifiers" }
}

init {
modifiersField.isAccessible = true
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
package org.utbot.framework

import mu.KotlinLogging
import org.utbot.common.PathUtil.toPath
import java.io.IOException

private val logger = KotlinLogging.logger {}

private val defaultUserTrustedLibrariesPath: String = "${utbotHomePath}/trustedLibraries.txt"
private const val userTrustedLibrariesKey: String = "utbot.settings.trusted.libraries.path"

object TrustedLibraries {
/**
* Always "trust" JDK.
*/
private val defaultTrustedLibraries: List<String> = listOf(
"java",
"sun",
"javax",
"com.sun",
"org.omg",
"org.xml",
"org.w3c.dom",
)

private val userTrustedLibraries: List<String>
get() {
val userTrustedLibrariesPath = System.getProperty(userTrustedLibrariesKey) ?: defaultUserTrustedLibrariesPath
val userTrustedLibrariesFile = userTrustedLibrariesPath.toPath().toFile()

if (!userTrustedLibrariesFile.exists()) {
return emptyList()
}

return try {
userTrustedLibrariesFile.readLines()
} catch (e: IOException) {
logger.info { e.message }

emptyList()
}
}

/**
* Represents prefixes of packages for trusted libraries -
* as the union of [defaultTrustedLibraries] and [userTrustedLibraries].
*/
val trustedLibraries: Set<String> by lazy { (defaultTrustedLibraries + userTrustedLibraries).toSet() }
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,16 @@ import kotlin.reflect.KProperty

private val logger = KotlinLogging.logger {}

/**
* Path to the utbot home folder.
*/
internal val utbotHomePath = "${System.getProperty("user.home")}/.utbot"

/**
* Default path for properties file
*/
internal val defaultSettingsPath = "${System.getProperty("user.home")}/.utbot/settings.properties"
internal const val defaultKeyForSettingsPath = "utbot.settings.path"
private val defaultSettingsPath = "$utbotHomePath/settings.properties"
private const val defaultKeyForSettingsPath = "utbot.settings.path"

internal class SettingDelegate<T>(val initializer: () -> T) {
private var value = initializer()
Expand Down Expand Up @@ -176,13 +181,22 @@ object UtSettings {
var enableMachineLearningModule by getBooleanProperty(true)

/**
* Options below regulate which NullPointerExceptions check should be performed.
* Options below regulate which [NullPointerException] check should be performed.
*
* Set an option in true if you want to perform NPE check in the corresponding situations, otherwise set false.
*/
var checkNpeInNestedMethods by getBooleanProperty(true)
var checkNpeInNestedNotPrivateMethods by getBooleanProperty(false)
var checkNpeForFinalFields by getBooleanProperty(false)

/**
* This option determines whether we should generate [NullPointerException] checks for final or non-public fields
* in non-application classes. Set by true, this option highly decreases test's readability in some cases
* because of using reflection API for setting final/non-public fields in non-application classes.
*
* NOTE: default false value loses some executions with NPE in system classes, but often most of these executions
* are not expected by user.
*/
var maximizeCoverageUsingReflection by getBooleanProperty(false)

/**
* Activate or deactivate substituting static fields values set in static initializer
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1044,7 +1044,7 @@ enum class MockStrategyApi(
override val displayName: String,
override val description: String
) : CodeGenerationSettingItem {
NO_MOCKS("No mocks", "Do not use Mock frameworks at all"),
NO_MOCKS("No mocks", "Do not use mock frameworks at all"),
OTHER_PACKAGES(
"Other packages: $MOCKITO",
"Mock all classes outside the current package except system ones"
Expand Down Expand Up @@ -1105,7 +1105,7 @@ enum class MockFramework(

enum class CodegenLanguage(
override val displayName: String,
@Suppress("unused") override val description: String = "Generating unit tests in $displayName"
@Suppress("unused") override val description: String = "Generate unit tests in $displayName"
) : CodeGenerationSettingItem {
JAVA(displayName = "Java"),
KOTLIN(displayName = "Kotlin");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ data class UtExecutionSuccess(val model: UtModel) : UtExecutionResult() {

sealed class UtExecutionFailure : UtExecutionResult() {
abstract val exception: Throwable
val isCheckedException get() = !(exception is RuntimeException || exception is Error)
}

data class UtOverflowFailure(
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
package org.utbot.framework.plugin.api.util

val Throwable.description
get() = message?.replace('\n', '\t') ?: "<Throwable with empty message>"

val Throwable.isCheckedException
get() = !(this is RuntimeException || this is Error)

val Class<*>.isCheckedException
get() = !(RuntimeException::class.java.isAssignableFrom(this) || Error::class.java.isAssignableFrom(this))
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package org.utbot.engine.overrides.collections;

import org.utbot.api.annotation.UtClassMock;

import static org.utbot.api.mock.UtMock.makeSymbolic;

@UtClassMock(target = java.util.AbstractCollection.class, internalUsage = true)
public abstract class AbstractCollection<E> implements java.util.Collection<E> {
@Override
public String toString() {
return makeSymbolic();
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
package org.utbot.engine.overrides.collections;

import org.utbot.api.annotation.UtClassMock;

import java.util.Collection;

@UtClassMock(target = java.util.List.class, internalUsage = true)
public interface List<E> extends java.util.List<E> {
static <E> java.util.List<E> of() {
return new UtArrayList<>();
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1) {
return new UtArrayList<>((E[]) new Object[]{e1});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2) {
return new UtArrayList<>((E[]) new Object[]{e1, e2});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5, E e6) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5, e6});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5, e6, e7});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7, E e8) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5, e6, e7, e8});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7, E e8, E e9) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5, e6, e7, e8, e9});
}

@SuppressWarnings("unchecked")
static <E> java.util.List<E> of(E e1, E e2, E e3, E e4, E e5, E e6, E e7, E e8, E e9, E e10) {
return new UtArrayList<>((E[]) new Object[]{e1, e2, e3, e4, e5, e6, e7, e8, e9, e10});
}

@SafeVarargs
@SuppressWarnings("varargs")
static <E> java.util.List<E> of(E... elements) {
return new UtArrayList<>(elements);
}

static <E> java.util.List<E> copyOf(Collection<? extends E> collection) {
return new UtArrayList<>(collection);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@
import org.utbot.engine.overrides.stream.UtStream;

import static org.utbot.api.mock.UtMock.assume;
import static org.utbot.api.mock.UtMock.assumeOrExecuteConcretely;
import static org.utbot.engine.ResolverKt.MAX_LIST_SIZE;
import static org.utbot.engine.overrides.UtOverrideMock.alreadyVisited;
import static org.utbot.api.mock.UtMock.assumeOrExecuteConcretely;
import static org.utbot.engine.overrides.UtOverrideMock.executeConcretely;
import static org.utbot.engine.overrides.UtOverrideMock.parameter;
import static org.utbot.engine.overrides.UtOverrideMock.visit;
Expand Down Expand Up @@ -63,6 +63,14 @@ public UtArrayList(Collection<? extends E> c) {
addAll(c);
}

public UtArrayList(E[] data) {
visit(this);
int length = data.length;
elementData = new RangeModifiableUnlimitedArray<>();
elementData.setRange(0, data, 0, length);
elementData.end = length;
}

/**
* Precondition check is called only once by object,
* if it was passed as parameter to method under test.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
import org.jetbrains.annotations.Nullable;

import static org.utbot.api.mock.UtMock.assume;
import static org.utbot.api.mock.UtMock.makeSymbolic;
import static org.utbot.engine.overrides.UtOverrideMock.alreadyVisited;
import static org.utbot.engine.overrides.UtOverrideMock.doesntThrow;
import static org.utbot.engine.overrides.UtOverrideMock.executeConcretely;
import static org.utbot.engine.overrides.UtOverrideMock.parameter;
import static org.utbot.engine.overrides.UtOverrideMock.visit;

Expand Down Expand Up @@ -556,8 +556,7 @@ public final boolean remove(Object o) {
// TODO rewrite it JIRA:1604
@Override
public String toString() {
executeConcretely();
return super.toString();
return makeSymbolic();
}

public final class Entry implements Map.Entry<K, V> {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
package org.utbot.engine.overrides.stream;

import org.utbot.api.annotation.UtClassMock;
import org.utbot.engine.overrides.collections.UtArrayList;

import java.util.List;
import java.util.stream.Stream;

@UtClassMock(target = java.util.Arrays.class, internalUsage = true)
Expand All @@ -16,5 +18,12 @@ public static <T> Stream<T> stream(T[] array, int startInclusive, int endExclusi
return new UtStream<>(array, startInclusive, endExclusive);
}

@SuppressWarnings({"unused", "varargs"})
@SafeVarargs
public static <T> List<T> asList(T... a) {
// TODO immutable collection https://github.com/UnitTestBot/UTBotJava/issues/398
return new UtArrayList<>(a);
}

// TODO primitive arrays https://github.com/UnitTestBot/UTBotJava/issues/146
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import org.utbot.engine.overrides.collections.UtHashSet
import org.utbot.engine.overrides.collections.UtLinkedList
import org.utbot.engine.pc.UtAddrExpression
import org.utbot.engine.pc.UtExpression
import org.utbot.engine.pc.UtFalse
import org.utbot.engine.pc.select
import org.utbot.engine.symbolic.asHardConstraint
import org.utbot.engine.z3.intValue
Expand Down Expand Up @@ -82,12 +83,24 @@ abstract class BaseOverriddenWrapper(protected val overriddenClassName: String)
return listOf(GraphResult(method.jimpleBody().graph()))
}

val overriddenMethod = overriddenClass.findMethodOrNull(method.subSignature)

val jimpleBody = overriddenMethod?.jimpleBody() ?: method.jimpleBody()
val graphResult = GraphResult(jimpleBody.graph())

return listOf(graphResult)
// We need to find either an override from the class (for example, implementation
// of the method from the wrapper) or a method from its ancestors.
// Note that the method from the ancestor might have substitutions as well.
// I.e., it is `toString` method for `UtArrayList` that is defined in
// `AbstractCollection` and has its own overridden implementation.
val overriddenMethod = overriddenClass
.findMethodOrNull(method.subSignature)
?.let { typeRegistry.findSubstitutionOrNull(it) ?: it }

return if (overriddenMethod == null) {
// No overridden method has been found, switch to concrete execution
pathLogger.warn("Method ${overriddenClass.name}::${method.subSignature} not found, executing concretely")
emptyList()
} else {
val jimpleBody = overriddenMethod.jimpleBody()
val graphResult = GraphResult(jimpleBody.graph())
listOf(graphResult)
}
}
}

Expand Down
Loading

0 comments on commit bf2e0e7

Please sign in to comment.