Update to Gradle 7.6 and JDK-19

[reta]

Signed-off-by: Andriy Redko andriy.redko@aiven.io

Description

Update to Gradle 7.6 and JDK-19

Issues Resolved

Closes #4637

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/5847/
• CommitID: db6789f
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/5848/
• CommitID: 3c445da
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[reta]

:-(

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/5855/
• CommitID: 8989808
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6178/
• CommitID: 0d4433d
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6179/
• CommitID: 0d4433d
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6209/
• CommitID: 9981b0f
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6215/
• CommitID: 9981b0f
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6292/
• CommitID: 2cc47de
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[reta]

Gradle Check (Jenkins) Run Completed with:

* **RESULT:** FAILURE x

* **URL:** https://build.ci.opensearch.org/job/gradle-check/6292/

* **CommitID:** [2cc47de](https://github.com/opensearch-project/OpenSearch/commit/2cc47dead0926577b12062432a35880e9849abe1)
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure [a flaky test](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#flaky-tests) unrelated to your change?

:o his is new

[Pipeline] End of Pipeline
java.lang.NullPointerException

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6294/
• CommitID: 2cc47de
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6297/
• CommitID: 2c57999
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6300/
• CommitID: 2c57999
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6319/
• CommitID: 02409f5
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[reta]

Impacted by https://bugs.openjdk.org/browse/JDK-8292027

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6320/
• CommitID: 3b87a37
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6358/
• CommitID: 3b87a37
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6361/
• CommitID: fa2192e
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6362/
• CommitID: d25b19c
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6535/
• CommitID: 0601206
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6536/
• CommitID: 0601206
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: FAILURE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6541/
• CommitID: 0601206
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6548/
• CommitID: 0601206
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: UNSTABLE ❌
• URL: https://build.ci.opensearch.org/job/gradle-check/6556/
• CommitID: 0601206
  Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
  Is the failure a flaky test unrelated to your change?

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/6563/
• CommitID: 0601206

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/6578/
• CommitID: 0167507

[github-actions]

Gradle Check (Jenkins) Run Completed with:

• RESULT: SUCCESS ✅
• URL: https://build.ci.opensearch.org/job/gradle-check/6581/
• CommitID: f276b0c

[dblock]

I want to make sure we're ok upgrading the bundled JDK to 19. I'm going to merge and we can always revert. But any downsides? Thoughts? cc: @nknize

[reta]

I want to make sure we're ok upgrading the bundled JDK to 19. I'm going to merge and we can always revert. But any downsides? Thoughts? cc: @nknize

@dblock I think @nknize is all in for JDK-19 (#4637 (comment))

[dblock]

I want to make sure we're ok upgrading the bundled JDK to 19. I'm going to merge and we can always revert. But any downsides? Thoughts? cc: @nknize

@dblock I think @nknize is all in for JDK-19 (#4637 (comment))

Now I am really concerned 🤣

[andrross]

I want to make sure we're ok upgrading the bundled JDK to 19

I have strong reservations about creating any OpenSearch release that bundles a non-LTS version of Java. I know it has been done in the past but it feels like we're setting users up to quickly be in a bad spot. However, I have no objections to integrating early with new Java features on the unstable main branch here.

[reta]

I want to make sure we're ok upgrading the bundled JDK to 19

I have strong reservations about creating any OpenSearch release that bundles a non-LTS version of Java. I know it has been done in the past but it feels like we're setting users up to quickly be in a bad spot. However, I have no objections to integrating early with new Java features on the unstable main branch here.

@andrross I share your concern, @dblock what we could do:

• revert bundled JDK to LTS (17)
• configure Gradle check against Java runtimes: 11/17/19 (we have all needed support now)

What do you think guys?

[dblock]

I am 50/50. On one hand main is ok to go fast and forward, but on the other hand maybe we're being too quick. I like the plan above.

[nknize]

I have strong reservations about creating any OpenSearch release that bundles a non-LTS version of Java.

Bundled for 3.0, yes. Not for 2.x. OpenSearch 3.0 will not be released until TBD next year, most likely 6 months (at the earliest) after JDK 19 stable (which was already released). The next LTS version is not planned until, jdk21? Non-LTS doesn't mean unstable so I don't think we need to succumb to FUD for bundling main w/ JDK 19 now.

/cc @uschindler @rmuir?, curious on your thoughts since y'all stay plugged in w/ the JDK community.

[rmuir]

we require java 17 for lucene, but we have some stuff that kicks in if you use 19, and pass --preview.

I don't understand the bundling of JDKs at all though: seems like a nightmare to include such a thing even if it is LTS version. Who patches it with critical security vulnerabilities etc? Do you release a new opensearch everytime jdk releases a new minor version? You are bundling it!

To me it seems better to just require a minimal version and leave the system package manager to the task of maintaining the JDK.

[uschindler]

I think that's inherited from the former codebase. I always try to download the zip of Elasticsearch or Opensearch without bundled JDK. But it gets harder to do that. Otherwise I do rm -rf

[reta]

I don't understand the bundling of JDKs at all though: seems like a nightmare to include such a thing even if it is LTS version. Who patches it with critical security vulnerabilities etc? Do you release a new opensearch everytime jdk releases a new minor version?

Yeah, it is inherited. We regularly update but AFAIK we do not align OpenSearch releases with JDK patch releases, we also have no-jdk bundles.

[nknize]

I think that's inherited from the former codebase.

That's right. I only vaguely recall conversations there about justifications for bundling (might've been after @rmuir time?). I seem to recall the reasons revolving around OS distribution / container (docker, kubernetes) control (e.g., having control over every dependency to support glibc / CentOS nightmares - or at least make debugging them easier).

I always try to download the zip of Elasticsearch or Opensearch without bundled JDK. But it gets harder to do that. Otherwise I do rm -rf

I think for self managed this is probably pretty common? Of course there is a large base that just uses containers (which I believe primarily uses the bundled jdk)

To me it seems better to just require a minimal version and leave the system package manager to the task of maintaining the JDK.

+1 , and probably leave the jdk version/config up to the user in the dockerfile? (I'm not a big container user so my knowledge is inch deep here)

[rmuir]

If i built a container for opensearch, let's say using centos as a base image, i still don't want a bundled JDK :) I would expect to just issue 'yum install -y opensearch-X-Y-Z' in my Dockerfile.

And I'd expect the opensearch package there not to contain a (Possibly outdated) JDK, but instead to have a package dependency, say on java-11-openjdk-headless or java-17-openjdk-headless or something like that: always with the latest bugfixes and security patches. I can rebuild container to have the latest stuff, no opensearch involvement needed.

Personally I think bundling JDKs is a big risk... a "bundled" JDK is like a "secret" JDK that exists outside of my package manager, never to get security updates etc. It isn't even obvious that it exists to someone doing a security audit or anything like that, they may miss it.

[dblock]

I am pretty sure that a bundled JDK was for "ease of use". I opened #5187 to continue discussing, and your thoughts would very much be appreciated. Particularly I'd like to understand in that issue what would break if we removed JDK bundling altogether (tar distribution would not work without a pre-installed JDK, but other than that? please add your comments to the issue).