Add functional tests for the GradleInspector #6882

sschuberth · 2023-04-19T11:51:49Z

Please have a look at the individual commit messages for the details.

sschuberth · 2023-04-19T11:56:11Z

...radle-inspector/src/funTest/assets/projects/synthetic/gradle-android-expected-output-app.yml

@@ -396,10 +396,10 @@ packages:
  description: "An HTTP+HTTP/2 client for Android and Java applications"
  homepage_url: "https://github.com/square/okhttp/okhttp"
  binary_artifact:
-    url: ""
+    url: "https://repo.maven.apache.org/maven2/com/squareup/okhttp3/okhttp/3.10.0/okhttp-3.10.0.jar"


The changes in this file are obviously to the better.

sschuberth · 2023-04-19T11:58:09Z

...-inspector/src/funTest/assets/projects/synthetic/gradle-expected-output-lib-without-repo.yml

 packages: []
+issues:


The GI reports these on the project level instead of per-scope, which probably makes sense as repositories are also defined on the project level. This implies some nice de-duplication.

I would say dependencies are defined on the scope level. What if the errors occur further down in the tree, would they also be reporter on the project level instead of inside the tree?

I would say dependencies are defined on the scope level.

True. What I meant was: Repositories are defined on the project level, so errors caused by no repository being present could be regarded as project configuration errors, and in that sense it makes sense (heh) to report the errors on the project level. However, missing repos are probably not the only cause for a dependency to be unresolved.

Probably the question boils down to: Is the position in the dependency tree important, i.e. may it have any influence on the error. I guess not.

But I'm undecided. I'm fine either way, to record unresolved dependencies (deduplicated) on the project level, or inside the dependency tree (with possible duplicates).

Probably the question boils down to: Is the position in the dependency tree important, i.e. may it have any influence on the error. I guess not.
But I'm undecided. I'm fine either way, to record unresolved dependencies (deduplicated) on the project level, or inside the dependency tree (with possible duplicates).

For direct dependencies this might not be so important, but for transitive dependencies it can be very helpful to see where in the tree the issue occurs. I would appreciate if we would report issues inside the tree where possible.

but for transitive dependencies it can be very helpful to see where in the tree the issue occurs.

Just out of curiosity, could you give an example where knowing the position in the tree is more helpful than having the issue at the top?

Looks like you're still fine for now with the current state as you've approved, so I'll eventually fix this up later to get the tests in now.

...-inspector/src/funTest/assets/projects/synthetic/gradle-expected-output-lib-without-repo.yml

sschuberth · 2023-04-19T11:59:54Z

...inspector/src/funTest/assets/projects/synthetic/multi-kotlin-project-expected-output-cli.yml

@@ -24,14 +24,6 @@ project:
      dependencies:
      - id: "Maven:org.jetbrains:annotations:13.0"
      - id: "Maven:org.jetbrains.kotlin:kotlin-stdlib-common:1.7.20"
-  - name: "implementationDependenciesMetadata"


GI generally skips "DependenciesMetadata" configurations / scopes.

sschuberth · 2023-04-19T12:00:11Z

...inspector/src/funTest/assets/projects/synthetic/multi-kotlin-project-expected-output-cli.yml

@@ -161,14 +113,6 @@ project:
      dependencies:
      - id: "Maven:org.jetbrains:annotations:13.0"
      - id: "Maven:org.jetbrains.kotlin:kotlin-stdlib-common:1.7.20"
-  - name: "testImplementationDependenciesMetadata"


GI generally skips "DependenciesMetadata" configurations / scopes.

sschuberth · 2023-04-19T12:00:21Z

...nspector/src/funTest/assets/projects/synthetic/multi-kotlin-project-expected-output-core.yml

@@ -22,12 +22,6 @@ project:
      dependencies:
      - id: "Maven:org.jetbrains:annotations:13.0"
      - id: "Maven:org.jetbrains.kotlin:kotlin-stdlib-common:1.7.20"
-  - name: "implementationDependenciesMetadata"


GI generally skips "DependenciesMetadata" configurations / scopes.

sschuberth · 2023-04-19T12:00:35Z

...nspector/src/funTest/assets/projects/synthetic/multi-kotlin-project-expected-output-core.yml

@@ -148,12 +105,6 @@ project:
      dependencies:
      - id: "Maven:org.jetbrains:annotations:13.0"
      - id: "Maven:org.jetbrains.kotlin:kotlin-stdlib-common:1.7.20"
-  - name: "testImplementationDependenciesMetadata"


GI generally skips "DependenciesMetadata" configurations / scopes.

sschuberth · 2023-04-19T12:03:39Z

...inspector/src/funTest/assets/projects/synthetic/multi-kotlin-project-expected-output-cli.yml

@@ -42,9 +34,6 @@ project:
      - id: "Maven:org.jetbrains.kotlin:kotlin-reflect:1.7.20"
        dependencies:
        - id: "Maven:org.jetbrains.kotlin:kotlin-stdlib:1.7.20"
-          dependencies:


kotlin-stdlib's dependencies were already seen on a level closer to the root (see line 49 and below), so they are skipped here. I'm not sure why the Gradle package manager implementation does not skip it, too. Similar for other cases below.

I'm not sure why the Gradle package manager implementation does not skip it, too.

That's because the "legacy" Gradle analyzer natively creates a graph which is then resolved back to a tree for test result comparison, and this expands the cycles again.

codecov · 2023-04-19T12:04:05Z

Codecov Report

Patch coverage has no change and project coverage change: -0.03 ⚠️

Comparison is base (4d0f4d2) 64.37% compared to head (8e99670) 64.35%.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #6882      +/-   ##
============================================
- Coverage     64.37%   64.35%   -0.03%     
  Complexity     1976     1976              
============================================
  Files           331      331              
  Lines         16729    16744      +15     
  Branches       2380     2385       +5     
============================================
+ Hits          10770    10775       +5     
- Misses         4919     4929      +10     
  Partials       1040     1040

Flag	Coverage Δ
funTest-docker	`59.03% <ø> (-0.15%)`	⬇️
funTest-non-docker	`29.58% <ø> (ø)`
test	`40.32% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 2 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

sschuberth · 2023-04-20T07:51:16Z

Note that I've extracted the required Android SDK upgrade to #6887.

plugins/package-managers/gradle-inspector/build.gradle.kts

Do not run the `funTest` tasks from the `gradle-inspector` and `gradle-package-manager` projects in parallel to avoid issues in `GradleAndroidFunTest` when dynamically installing missing Android SDK components, as both Gradle projects will start to share the same test projects in an upcoming change. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

sschuberth · 2023-05-23T17:04:31Z

@mnonnenmacher @fviernau this is up for review again as I've found the cause for the failing GradleAndroidFunTest: The test must not be run in parallel for the gradle-package-manager and gradle-inspector projects as the Android component installation triggered by the Android Gradle plugin might fail then.

Copy functional tests and expected results (but not the test projects) from the `Gradle` package manager implementation and adjust tests so that they run, but not necessarily pass yet. A later commit will adjust the expected results to show the differences with the `GradleInspector` implementation. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

Update expected results from the `Gradle` package manager to match those for the `GradleInspector` package manager. This highlights the differences between the two implementations. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

Do not cut the dependency tree on any already visited dependency in a configuration, but only for those that appear on the path from the root to a leaf. This fixes the discrepancy compared to the `Gradle` package manager that was identified at [1] but justified wrongly. [1]: #6882 (comment) Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

sschuberth requested a review from a team as a code owner April 19, 2023 11:51

sschuberth commented Apr 19, 2023

View reviewed changes

...-inspector/src/funTest/assets/projects/synthetic/gradle-expected-output-lib-without-repo.yml Outdated Show resolved Hide resolved

sschuberth commented Apr 19, 2023

View reviewed changes

sschuberth force-pushed the gi-fun-tests branch from 9b6a7a2 to 9df8df5 Compare April 20, 2023 06:39

sschuberth force-pushed the gi-fun-tests branch from 9df8df5 to fbc791a Compare April 20, 2023 20:38

sschuberth requested a review from tsteenbe as a code owner April 20, 2023 20:38

sschuberth requested a review from mnonnenmacher April 20, 2023 20:38

github-advanced-security bot found potential problems Apr 20, 2023

View reviewed changes

sschuberth force-pushed the gi-fun-tests branch from fbc791a to 8ab1a3a Compare April 20, 2023 21:07

sschuberth marked this pull request as draft April 21, 2023 07:54

sschuberth force-pushed the gi-fun-tests branch 3 times, most recently from 7e083e3 to 65547d9 Compare April 27, 2023 20:45

sschuberth force-pushed the gi-fun-tests branch from 65547d9 to 2f40bf9 Compare May 23, 2023 12:56

sschuberth force-pushed the gi-fun-tests branch 2 times, most recently from 57c8bd7 to e431d15 Compare May 23, 2023 14:40

sschuberth marked this pull request as ready for review May 23, 2023 17:01

sschuberth added 3 commits May 23, 2023 19:14

test(gradle-inspector): Disable Gradle tests that do not work yet

0bcbec7

Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

test(gradle-inspector): Adjust expected results

8e99670

Update expected results from the `Gradle` package manager to match those for the `GradleInspector` package manager. This highlights the differences between the two implementations. Signed-off-by: Sebastian Schuberth <sschuberth@gmail.com>

sschuberth force-pushed the gi-fun-tests branch from e431d15 to 8e99670 Compare May 23, 2023 17:14

mnonnenmacher approved these changes May 23, 2023

View reviewed changes

sschuberth merged commit 9c3cef6 into main May 23, 2023

sschuberth deleted the gi-fun-tests branch May 23, 2023 21:09

sschuberth mentioned this pull request Sep 10, 2024

Make GradleInspector the new default analyzer for Gradle projects #9070

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add functional tests for the GradleInspector #6882

Add functional tests for the GradleInspector #6882

sschuberth commented Apr 19, 2023

sschuberth Apr 19, 2023

sschuberth Apr 19, 2023

mnonnenmacher Apr 20, 2023

sschuberth Apr 20, 2023

mnonnenmacher May 23, 2023

sschuberth May 23, 2023

sschuberth May 23, 2023

sschuberth Apr 19, 2023

sschuberth Apr 19, 2023

sschuberth Apr 19, 2023

sschuberth Apr 19, 2023

sschuberth Apr 19, 2023 •

edited

Loading

sschuberth Apr 20, 2023

codecov bot commented Apr 19, 2023 •

edited

Loading

sschuberth commented Apr 20, 2023

sschuberth commented May 23, 2023

		packages: []
		issues:

Add functional tests for the GradleInspector #6882

Add functional tests for the GradleInspector #6882

Conversation

sschuberth commented Apr 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sschuberth Apr 19, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov bot commented Apr 19, 2023 • edited Loading

Codecov Report

sschuberth commented Apr 20, 2023

sschuberth commented May 23, 2023

sschuberth Apr 19, 2023 •

edited

Loading

codecov bot commented Apr 19, 2023 •

edited

Loading