Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradle support for secondary versions.props file #8017

Closed
tpittmanVS opened this issue Dec 15, 2020 · 15 comments · Fixed by #19182
Closed

Gradle support for secondary versions.props file #8017

tpittmanVS opened this issue Dec 15, 2020 · 15 comments · Fixed by #19182
Labels
manager:gradle Gradle package manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started

Comments

@tpittmanVS
Copy link

In addition to the typical build.gradle file, we store version info in versions.props as described here https://github.com/palantir/gradle-consistent-versions. We need versions.props tracked and incremented in the same way as renovatebot does for build.gradle

Is this feature currently supported or in the product roadmap?

@rarkins
Copy link
Collaborator

rarkins commented Dec 15, 2020

I know we certainly don't support the versions.lock part, so the answer is probably No. Can you provide an example repository and elaborate on the requirements further? e.g. is this completely defined by the palantir plugin only or is that also leveraging some other plugins or Gradle concepts Eg for the lock file?

@rarkins rarkins added manager:gradle Gradle package manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others auto:reproduction A minimal reproduction is necessary to proceed labels Dec 15, 2020
@tpittmanVS
Copy link
Author

Hey Rhys, thanks for your quick response. I think this simple example captures our use case: https://github.com/palantir/conjure-java-example. I believe gradle substitutes values from top-level versions.props when evaluating build.gradle.

As far as implementation details, I'm not too sure as I'm new to Gradle. I know the whole consistent versions/versions.prop framework does use Gradle Constraints. So to my eye, Gradle and Gradle Wrapper (gradlew) seem to be the only dependencies, but I have never set it up personally

For your own information, we use this framework across all our repos, and no one in the market seems to support it (dependabot, renovatebot, snyk, etc). We are considering whether it might be worth it to submit a PR to implement this functionality ourselves

@viceice
Copy link
Member

viceice commented Dec 15, 2020

I think we should add a separate manager for handling that file, as it's easy to parse and update by renovate.
The only part we need todo is running gradle to update the lockfile. This manager can support lockfile only. 🙃

Maybe we can name it gradle-consistent-versions. I'm very interested in this, as it would make dependency updating for renovate and gradle very easy. 🎉

@rarkins rarkins added the status:requirements Full requirements are not yet known, so implementation should not be started label Jan 12, 2021
@HonkingGoose

This comment has been minimized.

@rarkins rarkins added reproduction:provided and removed auto:reproduction A minimal reproduction is necessary to proceed labels Mar 3, 2021
@github-actions
Copy link
Contributor

github-actions bot commented Mar 3, 2021

Thank you for providing a reproduction! 🎉 🚀

The Renovate team will take a look at the reproduction repository. Once we confirm the provided repository reproduces the problem, the label will be changed to reproduction:confirmed.

@janhoy
Copy link
Contributor

janhoy commented Nov 6, 2022

Any status on this? I plan to attempt making a new manager. It would need to

  • Parse versions.props and versions.lock
  • Filter versions.lock against versions.props, leaving only the entries that match at least one props line (may be wildcard expr), leaving our direct and not transitive dependencies.
  • Parse versions and lookup upgrades
  • The PR needs to be made against versions.props, and run gradlew —write-locks before committing

@viceice
Copy link
Member

viceice commented Nov 6, 2022

Any status on this? I plan to attempt making a new manager. It would need to

  • Parse versions.props and versions.lock
  • Filter versions.lock against versions.props, leaving only the entries that match at least one props line (may be wildcard expr), leaving our direct and not transitive dependencies.
  • Parse versions and lookup upgrades
  • The PR needs to be made against versions.props, and run gradlew —write-locks before committing

no new manager. should be implemented inside Gradle manager.

@viceice
Copy link
Member

viceice commented Nov 6, 2022

@janhoy
Copy link
Contributor

janhoy commented Nov 25, 2022

Is this unblocked due to #18663 ?

@rarkins
Copy link
Collaborator

rarkins commented Nov 25, 2022

Yes no longer blocked

@janhoy
Copy link
Contributor

janhoy commented Nov 27, 2022

Ok, I'm attempting a PR to add support for this plugin.

@janhoy
Copy link
Contributor

janhoy commented Nov 29, 2022

I now have a working branchthat produces correct PRs for this plugin, see https://github.com/janhoy/gradle-consistent/pulls

I'll do some cleanups and publish a PR...

@janhoy
Copy link
Contributor

janhoy commented Nov 29, 2022

Another semi-related question:

In one project that I plan to start using renovate with this feature, the automated PR validation process also includes validating SHA-sum and licenses for all dependencies, i.e. we "lock" not only the dependency version, but also the dependency JAR SHA-sum in the repository.

So when renovate opens a PR, it will patch versions.props, update versions.lock (through --write-locks), but we also need it to run an arbitrary gradle task to compute new checksums. I see from the docs that there is postUpgradeTasks for self-hosted instances of renovate, due to security reasons of course. We'd like to use the official version, but I cannot find a way to specify an additional gradle task to run as part of the pre-commit update.

Before I open a new feature request issue, I thought I'd discuss it in context here. I'm thinking there could be a way for a repo to specify in renovate.json an additional gradle task, e.g.

{
  "gradle": {
    "postUpgradeGradleTasks": ["task1", "task2"], 
    "postUpgradeFileFilters": ["shasums.txt"],
  }
}

Techinically we already run ./gradlew --write-locks or update-locks, so it could be appended to the same command, but would there be security concerns or other concerns? If you will consider it, I'll open a new issue for it.

@viceice
Copy link
Member

viceice commented Nov 29, 2022

yes, there is a concern, as the task can manipulate the git directory/ hooks or the shared cache folder. so we don't like to allow arbitrary commands.

janhoy added a commit to janhoy/renovate that referenced this issue Nov 29, 2022
Fixes renovatebot#8017

Signed-off-by: Jan Høydahl <janhoy@users.noreply.github.com>
rarkins pushed a commit that referenced this issue Dec 8, 2022
Co-authored-by: Johannes Feichtner <Churro@users.noreply.github.com>
Closes #8017
@renovate-release
Copy link
Collaborator

🎉 This issue has been resolved in version 34.52.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
manager:gradle Gradle package manager priority-3-medium Default priority, "should be done" but isn't prioritised ahead of others status:requirements Full requirements are not yet known, so implementation should not be started
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants