Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot exclude kotlin-stdlib-jdk8 in 0.46.0 #733

Open
emilv opened this issue Feb 23, 2023 · 10 comments
Open

Cannot exclude kotlin-stdlib-jdk8 in 0.46.0 #733

emilv opened this issue Feb 23, 2023 · 10 comments

Comments

@emilv
Copy link

emilv commented Feb 23, 2023

After upgrade to the latest version 0.46.0 we get this error:

The following dependencies have later milestone versions:
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8 [1.8.10 -> 1.8.20-Beta]
     https://kotlinlang.org/

Here is our normal way of excluding versions, inspired by your README:

        fun isNonStable(candidate: ModuleComponentIdentifier): Boolean {
            val version = candidate.version
            val stableKeyword = listOf("FINAL", "GA", "RELEASE").any { version.toUpperCase().contains(it) }
            val regex = "^[0-9,.v-]+(-r)?$".toRegex()
            val isStable = stableKeyword || regex.matches(version)
            return isStable.not().also { notStable ->
                if (notStable) {
                    logger.log(
                        LogLevel.WARN,
                        "Excluding ${candidate.module}:${candidate.version} from dependency check because: Not stable",
                    )
                }
            }
        }
        
        rejectVersionIf {
            isNonStable(candidate.version)
        }

The isNonStable check works in 0.45.0 (without the explicit exclude in excludeKotlinVersions), but not in 0.46.0.

It does trigger the warning in both 0.45.0 and 0.46.0:

Excluding kotlin-stdlib-jdk8:1.8.20-Beta from dependency check because: Not stable
Excluding kotlin-stdlib-jdk8:1.8.20-Beta from dependency check because: Not stable

But I still get the version warning:

The following dependencies have later milestone versions:
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8 [1.8.10 -> 1.8.20-Beta]
     https://kotlinlang.org/
@ben-manes
Copy link
Owner

I tried it on Groovy and Kotlin, and it worked for me. Can you make a minimal reproduction?

Kotlin
import com.github.benmanes.gradle.versions.updates.DependencyUpdatesTask

plugins {
    `java-library`
    id("com.github.ben-manes.versions") version "0.46.0"
}

repositories {
  mavenCentral()
}

fun String.isNonStable(): Boolean {
  val stableKeyword = listOf("RELEASE", "FINAL", "GA").any { toUpperCase().contains(it) }
  val regex = "^[0-9,.v-]+(-r)?$".toRegex()
  val isStable = stableKeyword || regex.matches(this)
  return isStable.not()
}

tasks.withType<DependencyUpdatesTask> {
  rejectVersionIf {
    candidate.version.isNonStable()
  }
}

dependencies {
  implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10")
}
ben: kotlin $ gradle dU

> Task :dependencyUpdates

------------------------------------------------------------
: Project Dependency Updates (report to plain text file)
------------------------------------------------------------

The following dependencies are using the latest milestone version:
 - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin:0.46.0
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10

Gradle release-candidate updates:
 - Gradle: [7.5.1 -> 8.0.1]

Generated report file build/dependencyUpdates/report.txt
Groovy
plugins {
  id "java-library"
  id "com.github.ben-manes.versions" version "0.46.0"
}

repositories {
  mavenCentral()
}

def isNonStable = { String version ->
  def stableKeyword = ['RELEASE', 'FINAL', 'GA'].any { it -> version.toUpperCase().contains(it) }
  def regex = /^[0-9,.v-]+(-r)?$/
  return !stableKeyword && !(version ==~ regex)
}

tasks.named("dependencyUpdates").configure {
  rejectVersionIf {
    isNonStable(it.candidate.version)
  }
}

dependencies {
  implementation 'org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10'
}
ben: groovy $ gradle dU

> Task :dependencyUpdates

------------------------------------------------------------
: Project Dependency Updates (report to plain text file)
------------------------------------------------------------

The following dependencies are using the latest milestone version:
 - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin:0.46.0
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10

Gradle release-candidate updates:
 - Gradle: [7.5.1 -> 8.0.1]

Generated report file build/dependencyUpdates/report.txt

BUILD SUCCESSFUL in 3s
1 actionable task: 1 executed

@Adriankhl
Copy link

I am also experiencing this problem. One potential cause is that instead of writing implementation("org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10"), we typically just add kotlin("jvm") to plugins and the dependency resolution did something wrong there.

I might be able to provide a minimal reproduction example, but not today 😄 .

@emilv
Copy link
Author

emilv commented Feb 27, 2023

@Adriankhl That was my instinct as well. I did not manage to reproduce it in a minimal example though. So this is pretty interesting. I will also try to get some time to dig into it this week, but feel free to troubleshoot on your end.

@Adriankhl
Copy link

Adriankhl commented Mar 1, 2023

@ben-manes I think this problem is android-related, here a minimal build:

build.gradle.kts:

import com.github.benmanes.gradle.versions.updates.DependencyUpdatesTask

plugins {
    id("com.android.application") version "7.4.2"
    kotlin("android") version "1.8.10"
    id("com.github.ben-manes.versions") version "0.46.0"
}

repositories {
    mavenCentral()
    google()
}

android {
    compileSdkVersion(33)

    dependencies {
        implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.5.0")
    }
}

tasks.withType<DependencyUpdatesTask> {
    gradleReleaseChannel = "current"

    fun isNonStable(version: String): Boolean {
        return listOf(
            "-alpha",
            "-beta",
            "-dev",
            "-rc",
        ).any {
            version.lowercase().contains(it)
        }
    }

    rejectVersionIf {
        // ignored jacoco: https://github.com/ben-manes/gradle-versions-plugin/issues/534
        (candidate.group == "org.jacoco") || isNonStable(candidate.version)
    }
}

settings.gradle.kts:

pluginManagement {
    repositories {
        gradlePluginPortal()
        google()
    }
}

Output:

> Task :dependencyUpdates

------------------------------------------------------------
: Project Dependency Updates (report to plain text file)
------------------------------------------------------------

The following dependencies are using the latest milestone version:
 - com.android.application:com.android.application.gradle.plugin:7.4.2
 - com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin:0.46.0
 - org.jetbrains.kotlin:kotlin-compiler-embeddable:1.8.10
 - org.jetbrains.kotlin:kotlin-klib-commonizer-embeddable:1.8.10
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.20-Beta
 - org.jetbrains.kotlin.android:org.jetbrains.kotlin.android.gradle.plugin:1.8.10
 - org.jetbrains.kotlinx:kotlinx-serialization-json:1.5.0

The following dependencies have later milestone versions:
 - org.jetbrains.kotlin:kotlin-stdlib-jdk8 [1.8.10 -> 1.8.20-Beta]
     https://kotlinlang.org/

Failed to determine the latest version for the following dependencies (use --info for details):
 - org.jacoco:org.jacoco.ant

Gradle current updates:
 - Gradle: [8.0.1: UP-TO-DATE]

Generated report file build/dependencyUpdates/report.txt

BUILD SUCCESSFUL in 10s
1 actionable task: 1 executed

@ben-manes
Copy link
Owner

Selection of org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.20-Beta rejected by component selection rule: Rejected by rejectVersionIf
...
Comparing dependency (current: org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.10, latest: 1.8.20-Beta)
Comparing dependency (current: org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.20-Beta, latest: 1.8.20-Beta)

When running the buildEnvironment and dependencies tasks, I see that there is a dependency constraint,

org.jetbrains.kotlin:kotlin-stdlib-jdk8:{strictly 1.8.10} -> 1.8.10 (c)

If the dependency is declared on multiple configurations at different versions, then we will list each upgrade opportunity. We see kotlin-stdlib-jdk8:1.8.20-Beta as both the version being used and elsewhere as an upgrade option. Yet the resolutions by component selection all disallow it.

As mentioned, this worked correctly in 0.45. There were 3 changes in v46 so the next step is to see which caused the regression by building the commits as locally installed plugins.

@ben-manes
Copy link
Owner

Hey @Vampire,

It seems that your changes in #725 / #731 caused this regression. I narrowed the cause to this code change that had confused us,

 /** Returns the version status of the configuration's dependencies. */
 private fun getStatus(
   coordinates: Map<Coordinate.Key, Coordinate>,
   resolved: Set<ResolvedDependency>,
   unresolved: Set<UnresolvedDependency>,
 ): Set<DependencyStatus> {
   val result = hashSetOf<DependencyStatus>()
   for (dependency in resolved) {
     val resolvedCoordinate = Coordinate.from(dependency.module.id)
     val originalCoordinate = coordinates[resolvedCoordinate.key]
     val coord = originalCoordinate ?: resolvedCoordinate
-     if (originalCoordinate == null && resolvedCoordinate.groupId != "null") {
-       project.logger.info("Skipping hidden dependency: $resolvedCoordinate")
-     } else {
-       val projectUrl = getProjectUrl(dependency.module.id)
-       result.add(DependencyStatus(coord, resolvedCoordinate.version, projectUrl))
-     }
+     val projectUrl = getProjectUrl(dependency.module.id)
+     result.add(DependencyStatus(coord, resolvedCoordinate.version, projectUrl))
   }

In the logs I see that it was a hidden dependency that was skipped,

Skipping hidden dependency: org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.8.20-Beta

In this case the originalCoordinate was null, implying that it was implicitly manifested itself into the resolution results. That might happen due to a default dependency, which we do not report because it is confusing to users who expect to only update those that they declared. I think you only wanted to remove the second clause and we could restore this part. Can you take a look and let us know what you think?

@Vampire
Copy link
Contributor

Vampire commented Mar 2, 2023

I was under the impression that it duplicates the code at com.github.benmanes.gradle.versions.updates.Resolver#getCurrentCoordinates:

    // Ignore undeclared (hidden) dependencies that appear when resolving a configuration
   coordinates.keys.retainAll(declared.keys)

Which was also supported by the fact, that no test failed when removing that if completely and you not complaining during the review.

So yeah, if the mentioned line is not sufficient to exclude those, probably the if with only the first condition needs to be re-added together with a test, sorry.

@Vampire
Copy link
Contributor

Vampire commented Mar 2, 2023

Btw. it might make sense to have some way to display those too, it having them a separate category. If some configuration has default dependencies it is usually provided so that you can manually change the dependency, for example because coordinates of the artifact changed or a fork should be used. Sometimes additionally an extension property like toolVersion is provided that can be used to set the version that is used for the default dependencies. So even for those default dependencies it would often (if not always) make sense to show when they are outdated.

@ben-manes
Copy link
Owner

It raises questions because users don’t understand what to do about it or where they came from. Here I guess because kotlin hard codes the config name that constraints apply to, the copy resolves to a different version than their actual build. It didn’t honor the resolution strategy for some reason which would also annoy people, which I don’t think is a bug in our code.

I end up using a custom configuration to add tool dependencies to so that it is shown on the report.

I tried using a partial if clause but the tests failed. It will take some experimenting to figure this out again and satisfy everyone’s desired behaviors.

@Adriankhl
Copy link

As a temporary fix, add this to the tasks.withType<DependencyUpdatesTask> block:

    outputFormatter {
        // temporary fix for: https://github.com/ben-manes/gradle-versions-plugin/issues/733
        outdated.dependencies.removeAll { isNonStable(it.available.milestone.orEmpty()) }

        val plainTextReporter = PlainTextReporter(
            project,
            revision,
            gradleReleaseChannel
        )
        plainTextReporter.write(System.out, this)
    }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants