Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17406: Introduce Version Catalogs #2646

Closed
wants to merge 24 commits into from

Conversation

malliaridis
Copy link
Contributor

@malliaridis malliaridis commented Aug 14, 2024

https://issues.apache.org/jira/browse/SOLR-17406

IMPORTANT Due to the extensive scope with the dependency updates made in this PR, a new PR without the updates was created, see #2706.

Description

With Gradle it is possible to manage versions and dependencies in gradle files via version catalogs. This allows us to cleanup dependency resolution and move various versions from across the project to a single file.

Solution

This PR introduces version catalogs and removes the palantir consistent versions plugin, since it is no longer needed. Additionally, the build module(s) are restructured and further changes from apache/lucene#13484 were applied to reflect the same behavior.

Tests

Minor adjustments have been made to some test files that fix some references (due to restructuring of build files), and besides that spotless and tidy are added to run on build files too.

Open TODOs

  • Resolve dependency resolution conflicts
  • Update documentation about dependencies
  • Update license and notice files of updated dependencies
  • Reapply forbidden APIs' signature logic in forbidden-apis.gradle (depends on Support commons-io 2.16 policeman-tools/forbidden-apis#249)
  • Review dependency bot for breaking functionality
  • Update Changelog file
  • Verify the validity of the steps for "Update Lucene prerelease" (ASF account needed)

Checklist

Please review the following and check all that apply:

  • I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
  • I have created a Jira issue and added the issue ID to my pull request title.
  • I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
  • I have developed this patch against the main branch.
  • I have run ./gradlew check.
  • I have added tests for my changes.
  • I have added documentation for the Reference Guide

Dependecy Updates and Related PRs

This PR updates multiple dependencies (minor and patch versions) to the most up-to-date versions. These updates do not include breaking changes and therefore are applied in this PR. This should simplify conflict resolution with versions.props for any upcoming changes (see below PRs). The corresponding license and notice files for each dependency that was changed have been updated as well. Some license and notice files of libraries from the same project were merged.

In order to run a successful gradlew check (with Java 17 for now) some of the dependency issues (see #2643) had to be fixed as well.

Major upgrades or upgrades that introduce breaking changes are not included and should be addressed in different PRs.

Relates to #2047
Relates to #2129
Relates to #2130
Relates to #2167
Relates to #2169
Relates to #2268
Relates to #2396
Relates to #2398
Relates to #2608
Relates to #2609
Relates to #2628
Relates to #2637
Relates to #2639
Relates to #2638
Relates to #2643
Relates to #2647

The implementation of the eclipse plugin throws
"Value is null" error when trying to sync project
without the palantir consistent versions plugin.

Either fix the error or remove the configuration.
…e-version-catalog

# Conflicts:
#	solr/solr-ref-guide/build.gradle
#	versions.lock
#	versions.props
Move buildSrc into a composite included build (build-infra). Expose a plugin with buildinfra extension.

See apache/lucene@4afae6a
…e-version-catalog

Apply new versions from versions.prop to version catalog.

# Conflicts:
#	versions.lock
#	versions.props
# Conflicts:
#	solr/prometheus-exporter/build.gradle
#	versions.lock
#	versions.props
@iamsanjay
Copy link
Contributor

If this change is intended for Solr 10, it should be tested and developed using Java 21 to ensure there are no hidden surprises. I have one PR that makes Solr compatible with Java 21, but we later decided to break it into parts to reduce the burden for the final PR. I will prepare it by cherry-picking all the latest changes I made So that we can test this change.

@malliaridis
Copy link
Contributor Author

@iamsanjay That would be great. We probably have to add the foojay-resolver-plugin too at some point.

This commit updates apache log4j version to 2.23.1 and applies a fix for LOG4J2-3609 on the test-framework module.
forbiddenapis supports commons-io up to 2.15.1, so exclude for now some of the signature logic.

Apache Rat has breaking changes between 0.15 and 0.16.1.
Updates all the license files, notice files and checksums
of the updated dependencies.

This commit merges a couple license and notice files
of modules that belong to the same project and use
the same license and notice file.
# Conflicts:
#	solr/licenses/grpc-core-1.66.0.jar.sha1
#	versions.lock
#	versions.props
@malliaridis
Copy link
Contributor Author

malliaridis commented Aug 18, 2024

I think it was a mistake adding f402a0b onwards in the same PR, even thought there were only minor or patch version updates. It turned way larger than expected. Even though, over 11K out of 13K lines of code are from the versions.lock file, so maybe it is not that bad?

Maybe splitting this PR before finalizing is better.

@dweiss
Copy link
Contributor

dweiss commented Aug 23, 2024

I would keep any upgrades from infrastructure changes separate. You can then diff/ compare the final artifacts and see what's changed before doing any updates.

@malliaridis
Copy link
Contributor Author

Yeah I agree. I already started the process of splitting the PR into two, one for infra and one for the dependency updates. I am facing a few version differences with non-conflicting dependencies that belong to the same group (e.g. grpc dependencies).

I may have to introduce new BOMs if available to properly sync the versions across all artifacts of the same group.

@malliaridis
Copy link
Contributor Author

This PR is closed in favor to #2706, which does not include all the dependency updates made here. They will be resolved in separate PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants