Make UtEnumConstModel and UtClassRefModel reference models #414 #611

dtim · 2022-07-29T13:47:21Z

Description

Historically UtEnumConstModel and UtClassRefModel have been processed not as other reference models, but in a special way, more like to primitive types. This approach leads to several problems, especially to class cast errors when processing generic collections with enums or class references as elements.

This commit makes UtEnumConstModel and UtClassRefModel subtypes of UtReferenceModel.

Concrete executor is modified to respect the identity of static fields to avoid rewriting enum values and Class<?> instances.
Special processing for enums is implemented. When a new enum value is created, or an Object is being cast to the enum type, static values for the enum class are initialized, and the set of hard constraint is added to require that the new instance has the same address and ordinal as any one of enum constants to implement reference equality for enums.
Corresponding changes in fuzzer model providers have been implemented.

Limitations (probably have been there for a long time, addressed separately):

anonymous nested classes associated with enum values are not fully supported yet: No tests are generated for methods in enums with nested anonymous classes #617 (maybe a codegen limitation);
enum constants are explicitly assigned to corresponding static fields in generated tests: Useless assignment of enum values to corresponding static fields in generated tests #618 (it seems to be a particular case of our general approach to static field initialization);
instances of enum classes are (unsuccessfully) generated via reflection by the concrete executor: Concrete executor tries to create enum instances by reflection and fails #619.

Fixes #414 (meta-issue)
Fixes #230
Fixes #300

Type of Change

Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

Automated Testing

All existing unit tests should pass.

New unit tests for enums have been added: org.utbot.examples.enums.ComplexEnumExamplesTest

Note: the test org.utbot.examples.enums.ComplexEnumExamplesTest#testFindState is currently disabled because of limited anonymous classes support.

Manual Scenario

To check that #230 is fixed: generate the test suite for the code example in issue #230 description. Test should be generated, no exceptions should be thrown.

To check that #300 is fixed: run the contest estimator with settings from issue #300 description. Generated file should compile, class name should be present in qualified enum constant names.

Checklist:

This is the author self-check list

The change followed the style guidelines of the UTBot project
Self-review of the code is passed
The change contains enough commentaries, particularly in hard-to-understand areas
New documentation is provided or existed one is altered
No new warnings
New tests have been added
All tests pass locally with my changes

Damtev

Logic LGTM, some issues about codestyle

Damtev · 2022-08-01T12:09:23Z

utbot-framework-api/src/main/kotlin/org/utbot/framework/plugin/api/Api.kt

+ * Get model id or null if id is null or the model has no id.
+ */
+fun UtModel.idOrNull(): Int? = when (this) {
+    is UtNullModel -> 0


Could you link this constant with org.utbot.engine.ResolverKt#NULL_ADDR somehow, please?

What do you think about moving the constant from Resolver into Api.kt?

utbot-framework/src/main/kotlin/org/utbot/engine/CollectionWrappers.kt

utbot-framework/src/main/kotlin/org/utbot/engine/Memory.kt

utbot-framework/src/main/kotlin/org/utbot/engine/Traverser.kt

utbot-framework/src/test/kotlin/org/utbot/examples/enums/ComplexEnumExamplesTest.kt

utbot-framework-api/src/main/kotlin/org/utbot/framework/plugin/api/Api.kt

utbot-sample/src/main/java/org/utbot/examples/enums/ComplexEnumExamples.java

sergeypospelov

The concrete part looks good to me.

sergeypospelov · 2022-08-03T09:03:01Z

utbot-framework/src/main/kotlin/org/utbot/engine/symbolic/SymbolicStateUpdate.kt

+import org.utbot.engine.UtNamedStore
+import org.utbot.framework.plugin.api.ClassId
+import org.utbot.framework.plugin.api.FieldId
+import org.utbot.framework.plugin.api.classId


Looks like these imports are redundant.

dtim · 2022-08-03T11:54:37Z

Updated UtModelVisitor and FieldStateVisitor to reflect model hierarchy change.

Historically `UtEnumConstModel` and `UtClassRefModel` have been processed not as other reference models, but in a special way, more like to primitive types. This approach leads to several problems, especially to class cast errors when processing generic collections with enums or class references as elements. This commit makes `UtEnumConstModel` and `UtClassRefModel` subtypes of `UtReferenceModel`. * Concrete executor is modified to respect the identity of static fields to avoid rewriting enum values and `Class<?>` instances. * Special processing for enums is implemented. When a new enum value is created, or an `Object` is being cast to the enum type, static values for the enum class are initialized, and the set of hard constraint is added to require that the new instance has the same address and ordinal as any one of enum constants to implement reference equality for enums. * Corresponding changes in fuzzer model providers have been implemented. * UtModelVisitor has been updated to reflect the new inheritance hierarchy.

A custom id generator interface hierarchy is used instead of `IntSupplier`: * `IdGenerator` that allows to create fresh identifiers, * `IdentityPreservingIdGenerator`: `IdGenerator` that can return the same id for the same object. A default implementation of `IdentityPreservingIdGenerator` is used in fuzzer. It uses reference equality for object comparison, that allows to create distinct models of equal object (in `equals` sense), and to always assign the same id to the same enum value.

Make UtEnumConstModel and UtClassRefModel reference models #414 Historically `UtEnumConstModel` and `UtClassRefModel` have been processed not as other reference models, but in a special way, more like to primitive types. This approach leads to several problems, especially to class cast errors when processing generic collections with enums or class references as elements. This commit makes `UtEnumConstModel` and `UtClassRefModel` subtypes of `UtReferenceModel`. * Concrete executor is modified to respect the identity of static fields to avoid rewriting enum values and `Class<?>` instances. * Special processing for enums is implemented. When a new enum value is created, or an `Object` is being cast to the enum type, static values for the enum class are initialized, and the set of hard constraint is added to require that the new instance has the same address and ordinal as any one of enum constants to implement reference equality for enums. * `UtModelVisitor` has been updated to reflect the new inheritance hierarchy. * Corresponding changes in the fuzzer have been implemented, including id generation refactoring. A custom id generator interface hierarchy is now used instead of `IntSupplier`: - `IdGenerator` that allows to create fresh identifiers, - `IdentityPreservingIdGenerator`: `IdGenerator` that can return the same id for the same object. A default implementation of `IdentityPreservingIdGenerator` is used in fuzzer. It uses reference equality for object comparison, that allows to create distinct models of equal object (in `equals` sense), and to always assign the same id to the same enum value.

This was referenced Jul 30, 2022

Useless assignment of enum values to corresponding static fields in generated tests #618

Closed

Concrete executor tries to create enum instances by reflection and fails #619

Closed

dtim force-pushed the dtim/414-reference-subtypes branch from 19214b2 to 06e56ad Compare July 30, 2022 22:21

dtim marked this pull request as ready for review July 30, 2022 22:22

dtim requested review from Damtev, Markoutte and sergeypospelov July 30, 2022 22:22

Damtev requested changes Aug 1, 2022

View reviewed changes

dtim force-pushed the dtim/414-reference-subtypes branch 3 times, most recently from 3de411a to 84c66ce Compare August 1, 2022 17:07

Damtev approved these changes Aug 2, 2022

View reviewed changes

sergeypospelov approved these changes Aug 3, 2022

View reviewed changes

dtim force-pushed the dtim/414-reference-subtypes branch 2 times, most recently from 54af0a3 to 410613a Compare August 3, 2022 11:23

dtim requested a review from sergeypospelov August 3, 2022 11:54

dtim force-pushed the dtim/414-reference-subtypes branch 2 times, most recently from f4959aa to e29923f Compare August 4, 2022 04:19

Markoutte approved these changes Aug 4, 2022

View reviewed changes

dtim added 3 commits August 4, 2022 12:02

A unit test to check that enum ids in recursive models are consistent

b11b184

dtim force-pushed the dtim/414-reference-subtypes branch from 374eb3c to b11b184 Compare August 4, 2022 09:03

Markoutte approved these changes Aug 4, 2022

View reviewed changes

dtim merged commit e23289b into main Aug 4, 2022

dtim deleted the dtim/414-reference-subtypes branch August 4, 2022 12:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Make UtEnumConstModel and UtClassRefModel reference models #414 #611

Make UtEnumConstModel and UtClassRefModel reference models #414 #611

Uh oh!

dtim commented Jul 29, 2022 •

edited

Loading

Uh oh!

Damtev left a comment

Uh oh!

Damtev Aug 1, 2022

Uh oh!

dtim Aug 1, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergeypospelov left a comment

Uh oh!

sergeypospelov Aug 3, 2022

Uh oh!

dtim commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Make UtEnumConstModel and UtClassRefModel reference models #414 #611

Make UtEnumConstModel and UtClassRefModel reference models #414 #611

Uh oh!

Conversation

dtim commented Jul 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of Change

How Has This Been Tested?

Automated Testing

Manual Scenario

Checklist:

Uh oh!

Damtev left a comment

Choose a reason for hiding this comment

Uh oh!

Damtev Aug 1, 2022

Choose a reason for hiding this comment

Uh oh!

dtim Aug 1, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergeypospelov left a comment

Choose a reason for hiding this comment

Uh oh!

sergeypospelov Aug 3, 2022

Choose a reason for hiding this comment

Uh oh!

dtim commented Aug 3, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

dtim commented Jul 29, 2022 •

edited

Loading