Detect renamed functions and global variables. #774

TimOrtel · 2022-07-04T12:57:31Z

This pull requests builds on top of my pull request about detection renamed local variables.

It implements an algorithm, that can detect renamed functions and global variables on an incremental run. The algorithm is non recursive.

The following changes have been made:

Moved eqF from compareCIL.ml to detectGlobals.ml
Removed logic that would just compare globals with matching names with my algorithm implemented in detectRenamedFunctions.ml
compareAST.ml is now fully functional, because it cannot have side effects when two wrong functions are compared. Previous side-effective hacks in compareAST have been moved out and are now applied after it is clear that the functions actually match.

The algorithm works like this (simplified):

Find functions and global variables with matching names.
Test these function pairs:
2.1. Foreach function pair detected in 1.: Compare the functions bodies of the functions.
2.2. If a function reports back that it depends on a rename of a global variable/function, those dependecies are checked in the next step.
2.3. Foreach dependency, check if they do not rely on additional dependencies themselves. If this assumption holds for all dependencies, the original function pair from 2.1 is marked as renamed, otherwise it is marked as changed.
Report back the results to compareCIL and apply the change data correctly.

I added test cases, but they are not verified automatically, meaning they have to be verified manually.

…nt statements with those in many places

# Conflicts: # src/incremental/compareCIL.ml # src/util/server.ml

…eturn type

michael-schwarz · 2023-03-21T10:16:45Z

If I understand correctly, this is now finished and could be merged after another round of reviews?

stilscher · 2023-03-21T10:25:12Z

Yes, another round of feedback would be great. I think I also still need to make sure that the new incremental cram tests are also automatically executed in the CI. Other than that I think it is mostly cleaned up and ready for review.

src/framework/constraints.ml

src/util/server.ml

tests/incremental/04-var-rename/01-rename_and_shuffle.t

src/incremental/compareAST.ml

src/incremental/compareCFG.ml

sim642 · 2023-03-22T09:31:14Z

I think I also still need to make sure that the new incremental cram tests are also automatically executed in the CI.

That probably needs a step like this in the CI workflows:

      - name: Test incremental cram
        run: opam exec -- dune runtest tests/incremental

src/incremental/compareCIL.ml

michael-schwarz · 2023-05-31T07:45:01Z

@stilscher should we aim to get this merged before we hit the 11 month anniversary of this PR next Monday?

stilscher · 2023-05-31T08:27:14Z

I thought it would be good to wait for an approval in the reviews before merging.

sim642

All the comments have been resolved, so this should be good to go.

TimOrtel · 2023-05-31T13:35:28Z

🎉

TimOrtel added 30 commits March 17, 2022 14:53

Added basic test cases for changed variable names.

4256428

Rename detection works for simple cases

27dd10f

Rename detection for method parameters, too

6bc0fca

Renaming of method params should work now.

ca6670b

Renaming of results does work for the log files.

c1e165c

Added multiple incremental runs test

d589278

Renamed local vars are now also shown in g2html.

9eb3f87

Added incremental aware print statements and replaced traditional pri…

d652715

…nt statements with those in many places

Renamed variable names are now displayed with their new name in g2html

08da3fb

Cleanup print statements and added some docu.

3a11fb9

Merge remote-tracking branch 'upstream/master' into dev-local-var-rename

26d0702

# Conflicts: # src/incremental/compareCIL.ml # src/util/server.ml

Renamed context to rename_mapping

b7fac89

Replaced rename mapping lists with Hashtbls for increased performance

abf871b

cleanup print statements and code.

4745a3f

Merge upstream master

7d702f7

cherry picked context -> rename mapping

c645c68

Replaced rename mapping lists with Hashtbls for increased performance

aed7a3a

Old locals are now renamed to the new local names.

9e95ddb

Fixed duplicate id tests

7e89ec2

Added some test cases

e4df2ca

Added first functions for rename detection.

2c86411

Replaced Hashtbl in compare functions with Map

182db42

CompareAST functions now propagate updated rename_mappings in their r…

11de365

…eturn type

eqF now returns the method rename dependencies

d900e7e

Implemented rename detection of method. Not tested.

49caf8c

Removed obsolete method and smaller changes

abef116

Implemented simple rename detection for methods

b9ad6db

Added more docu

1855a5a

Added more test cases for method rename

c71e29c

Added json and patch files for method rename tests.

874519c

make semgrep happy

6ba71fc

michael-schwarz requested review from jerhard and sim642 and removed request for sim642 and jerhard March 21, 2023 12:02

sim642 reviewed Mar 22, 2023

View reviewed changes

stilscher added 6 commits March 23, 2023 10:54

add incremental cram tests to CI

aeaf301

refine implementation based on review comments

033555a

missing propagation of change_info and final_matches

e293a7f

remove test script because of restricted parser and few working cases

e8ce672

no structural comparison when looking through collected final_matches

af79fb2

fix check whether already matched as unchanged

7d68cf2

sim642 reviewed Mar 27, 2023

View reviewed changes

src/incremental/compareCIL.ml Outdated Show resolved Hide resolved

src/incremental/compareCIL.ml Outdated Show resolved Hide resolved

stilscher added 2 commits April 4, 2023 18:34

fix: already matched check needs to consider change_info

79873fe

remove polymorphic comparison of global_col

06502b5

stilscher requested a review from sim642 April 27, 2023 11:49

stilscher added 3 commits May 2, 2023 19:28

fix new and old global mix-up

61d15d1

handle functions without definitions in exact comparison

72065a5

Merge branch 'master' into dev-method-rename-simple-impl

2a8bce5

sim642 approved these changes May 31, 2023

View reviewed changes

stilscher merged commit 7379f9e into goblint:master May 31, 2023

sim642 added this to the v2.2.0 milestone Sep 11, 2023

sim642 mentioned this pull request Sep 13, 2023

[new release] goblint (2.2.0) ocaml/opam-repository#24420

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Detect renamed functions and global variables. #774

Detect renamed functions and global variables. #774

TimOrtel commented Jul 4, 2022

michael-schwarz commented Mar 21, 2023

stilscher commented Mar 21, 2023

sim642 commented Mar 22, 2023

michael-schwarz commented May 31, 2023

stilscher commented May 31, 2023

sim642 left a comment

TimOrtel commented May 31, 2023

Detect renamed functions and global variables. #774

Detect renamed functions and global variables. #774

Conversation

TimOrtel commented Jul 4, 2022

michael-schwarz commented Mar 21, 2023

stilscher commented Mar 21, 2023

sim642 commented Mar 22, 2023

michael-schwarz commented May 31, 2023

stilscher commented May 31, 2023

sim642 left a comment

Choose a reason for hiding this comment

TimOrtel commented May 31, 2023