Improve support for different Java versions #8249

mnonnenmacher · 2024-02-09T15:07:41Z

Currently the Docker image and the artifacts published to Maven Central are both built with Java 17. This can cause two kinds of issues:

Docker: As the Maven and Gradle analyzers are built into ORT and compiled with the same Java version, sometimes projects cannot be analyzed that require a different Java version. For example, the project could use a Gradle version that is not compatible with Java 17, or the project could require Java 21.
Maven Central: Projects that use ORT programmatically as a library, like the ORT workbench, must be compiled with at least the same Java version as the ORT artifacts. However, for the same reason as above it could be required to build those tools with a different Java version.

One approach to solve this would be to release the Docker images and the Maven Central artifacts with different variants for each of the Java LTS releases, currently relevant are Java 11, 17, and 21.

Another more complex approach would be to decouple the Maven and Gradle analyzers from ORT itself and release them as standalone tools, comparable to the python-inspector. These tools could then be released with a bundled JVM for the different Java versions.

Also see some related PRs and issues: #4948, #6311, #8027, #8242

heliocastro · 2024-02-11T15:41:34Z

I can say decoupling Maven, and Gradle is the best suitable solution.

Explain what would be the alternatives and how this can become a nightmare:

1 - Multiple docker images with all LTS images.
Today we build docker itself with Temurin as base: FROM eclipse-temurin:$JAVA_VERSION-jdk-$UBUNTU_VERSION as base
This means that we will need to build base as standard image, and take care of java installs on later stage to avoid blowup the images with 3x all layers.
We would treat java as another independent layer. Maintenance will be hard as long we start to change base distro, as we have layers pointing out for the base. Example:
Today we have all build over Ubuntu 22.04, and python 3.10.x build for this.
We will have soon the new LTS from Ubuntu, So base image will be updated to 24.04, and then same python 3.10.x will be built over this one. Any older image referring to this python will be unstable, as will point originally to older base.
And not forgetting, we need rename the images for each version.
Resume:
maintenance complexity: Higher, size increase: High, effectiveness: medium

2 - Use SDKMAN for java
So, instead of multiple images, we use SDKMAN to install parallel Java sdk's and during process change the environment to select the required JDK.
It will slightly increase our build time for docker, but then will make multiple Java SDK's install easier and is future proof.
But then we now transfer to Ort how it will manage in runtime the switch for the specific JDK and I honestly can't tell if is a good idea or not since I don't know that Ort code at all. Maybe a runtime environment switch will be easier from the Ort point of view.
Resume:
maintenance complexity: Low, size increase: Medium, effectiveness: ?

sschuberth · 2024-02-13T16:12:27Z

I'm also thinking into the direction of simply installing multiple JDK versions in parallel in the Docker image, and finding a way to choose the JDK at runtime for the package manager implementations. At least for SBT that would work, as it's called as an external CLI tool. Also for Gradle projects that leverage the toolchains mechanism that should work. Not sure about Maven, though.

mnonnenmacher · 2024-02-13T16:23:43Z

Not sure about Maven, though.

I'm not sure if have seen the issue with Maven projects so far, probably because it's more declarative than Gradle. Maybe it can happen if plugins are used the require a specific Java version, we at least load Wagon transport plugins.

sschuberth · 2024-02-28T11:21:25Z

My current idea, at least for SBT for now, is to bootstrap the required JDK via https://github.com/foojayio/discoclient at ORT analyzer runtime.

fb33 · 2024-03-07T15:20:06Z

Hi,
Just a feedback with jdk21.
I have some projects with gradle 8.5 and jdk21, and of course with the current ORT docker image, the analyzer failed.
I make a quick (and naive) test, building in ORT image with jdk21 and run the analyzer.
=> there is no more issue, but the analyzer found no dependency !

I have also some projects with maven and jdk21, and there is no issue with them, all is fine.

fb33 · 2024-03-08T09:38:33Z

@sschuberth
here is a sample of gradle project on jdk21 with the results&logs of ORT
https://github.com/fb33/sample-gradle8.5-jdk21

hope this help.
regards

Configure Gradle to build class files compatible with Java 11. This allows to use the published ORT artifacts also in Projects that use Java 11. Previously, Java 17 was required. The main use case for this is to build a Docker image with Java 11 to be able to analyze Gradle projects that are not compatible with newer Java versions. Relates to #8249. Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.com>

Configure Gradle to build class files compatible with Java 11. This allows to use the published ORT artifacts also in Projects that use Java 11. Previously, Java 17 was required. The main use case for this is to be able to build the Analyzer Docker image for the ORT Server [1] with Java 11 which is required to analyze Gradle projects that are not compatible with newer Java versions. Relates to #8249. [1]: https://github.com/eclipse-apoapsis/ort-server Signed-off-by: Martin Nonnenmacher <martin.nonnenmacher@bosch.com>

The previous code was calling `JavaVersion.current()` at build time, assuming that the Java version running Gradle is the same Java version used for building ORT. This assumption makes it more difficult to built ORT for different Java versions. Solve that by using Gradle's toolchains mechanism [1] instead. Choose Java 11 as the default for the new `javaLanguageVersion` property to support analyzing older projects with ORT, also see [2]. [1]: https://docs.gradle.org/current/userguide/toolchains.html [2]: #8249 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

The previous code was calling `JavaVersion.current()` at build time, assuming that the Java version running Gradle is the same Java version used for building ORT. This assumption makes it more difficult to build ORT for different Java versions in the same environment. Solve that by using Gradle's toolchains mechanism [1] instead. Choose Java 11 as the default for the new `javaLanguageVersion` property to support analyzing older projects with ORT, also see [2]. [1]: https://docs.gradle.org/current/userguide/toolchains.html [2]: #8249 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

Note that `setJavaHome()` [1] set the Java home to "to use for the Gradle process". However, as by default (unless toolchains are used [2]) "Gradle will compile Java code to the language level of the JVM running Gradle" [3], this will impact the Java target of the build. See #8249 for context. [1]: https://docs.gradle.org/current/javadoc/org/gradle/tooling/ConfigurableLauncher.html#setJavaHome(java.io.File) [2]: https://docs.gradle.org/8.10/userguide/building_java_projects.html#sec:compiling_with_toolchain [3]: https://docs.gradle.org/8.10/userguide/building_java_projects.html#sec:java_cross_compilation Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

Note that `setJavaHome()` [1] sets the Java home "to use for the Gradle process". However, as by default (unless toolchains are used [2]) "Gradle will compile Java code to the language level of the JVM running Gradle" [3], this will impact the Java target of the build. See #8249 for context. [1]: https://docs.gradle.org/current/javadoc/org/gradle/tooling/ConfigurableLauncher.html#setJavaHome(java.io.File) [2]: https://docs.gradle.org/8.10/userguide/building_java_projects.html#sec:compiling_with_toolchain [3]: https://docs.gradle.org/8.10/userguide/building_java_projects.html#sec:java_cross_compilation Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

As fcfab20 decoupled the Java version to use for analysis of Gradle projects from the Java version running ORT, finally update the Java version from 11 / 17 to 21. See [1] for context. [1]: #8249 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

As fcfab20 decoupled the Java version to use for analysis of Gradle projects from the Java version running ORT, finally update the Java version from 11 / 17 to 21. See [1] for context. Also stop using the deprecated `Thread.id` in favor of `Thread.threadId()` and ensure to run with Java 21 when generating completion scripts. [1]: #8249 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

sschuberth · 2024-09-20T14:08:22Z

Do you have any idea when you'll fix the usage of jdk21 for gradle and sbt project ?

@fb33 for Gradle it was basically fixed by #9127, for SBT it will be fixed by #9179. Both introduce package manager options to customize the JDK to use.

here is a sample of gradle project on jdk21 with the results&logs of ORT
https://github.com/fb33/sample-gradle8.5-jdk21

Could you yourself give #9179 a try on it?

fb33 · 2024-09-20T14:28:22Z

Hi @sschuberth

I have done some successful tests with ORT-32.1.0 👍

I don't know if I did it in the way you imagine, but here is what I done :

I built a special ORT docker image (with only tools for my stacks) in which I added open jdk 21.
In the ort.yml, I indicate the javaHome here :

    packageManagers:
      GradleInspector:
        options:
          javaHome: /usr/lib/jvm/java-21-openjdk-amd64

I override javaHome (the default one - 17, or the jdk21) on the fly according to the stack of projects being audited.

sschuberth · 2024-09-20T14:32:54Z

I override javaHome (the default one - 17, or the jdk21) on the fly according to the stack of projects being audited.

Yes, that's a perfectly valid way to do it. If you can limit the number of required JDKs beforehand, you can add them to the Docker image and just configure the right Java home per ORT run. Alternatively, if you don't know which JDK will be required, you can let ORT bootstrap a JDK at runtime that matches the javaVersion option. If both javaHome and javaVersion are given, javaVersion takes precedence.

fb33 · 2024-09-20T15:16:37Z

Great ! I'll try the bootstrap method.
thanks for the advice !

and good job for this new feature ! 👏

As fcfab20 decoupled the Java version to use for analysis of Gradle projects from the Java version running ORT, finally update the Java version from 11 / 17 to 21. See [1] for context. Also stop using the deprecated `Thread.id` in favor of `Thread.threadId()` and ensure to run with Java 21 when generating completion scripts. [1]: #8249 Signed-off-by: Sebastian Schuberth <sebastian@doubleopen.org>

sschuberth · 2024-10-09T13:29:41Z

@mnonnenmacher, do you think this can be closed now with the recent additions to customize the Java home / version?

mnonnenmacher · 2024-10-09T13:59:34Z

@mnonnenmacher, do you think this can be closed now with the recent additions to customize the Java home / version?

Yes.

mnonnenmacher added the analyzer About the analyzer tool label Feb 9, 2024

mnonnenmacher mentioned this issue Feb 9, 2024

Bumping all java versions to 21 #8242

Closed

sschuberth added the enhancement Issues that are considered to be enhancements label Feb 11, 2024

This comment has been minimized.

Sign in to view

mnonnenmacher mentioned this issue Mar 12, 2024

build: Set the target JVM to 11 #8403

Closed

sschuberth mentioned this issue Apr 16, 2024

Exception when using GradleInspector with Java 21 and Gradle<8.2 #8027

Closed

sschuberth mentioned this issue Jul 17, 2024

deps: update jirarestclient to v6 (major) #8859

Closed

1 task

sschuberth mentioned this issue Sep 6, 2024

feat(gradle-inspector): Allow to customize the Java home for analysis #9086

Merged

sschuberth mentioned this issue Sep 13, 2024

Build ORT with Java 21 #9134

Merged

mnonnenmacher closed this as completed Oct 9, 2024

janek64 mentioned this issue Oct 15, 2024

LocationAwareException when using GradleInspector with Gradle version < 6.8 #9282

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve support for different Java versions #8249

Improve support for different Java versions #8249

mnonnenmacher commented Feb 9, 2024 •

edited

Loading

heliocastro commented Feb 11, 2024

sschuberth commented Feb 13, 2024

mnonnenmacher commented Feb 13, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

sschuberth commented Feb 28, 2024

fb33 commented Mar 7, 2024 •

edited

Loading

This comment has been minimized.

This comment has been minimized.

fb33 commented Mar 8, 2024

sschuberth commented Sep 20, 2024 •

edited

Loading

fb33 commented Sep 20, 2024

sschuberth commented Sep 20, 2024

fb33 commented Sep 20, 2024

sschuberth commented Oct 9, 2024

mnonnenmacher commented Oct 9, 2024

Improve support for different Java versions #8249

Improve support for different Java versions #8249

Comments

mnonnenmacher commented Feb 9, 2024 • edited Loading

heliocastro commented Feb 11, 2024

sschuberth commented Feb 13, 2024

mnonnenmacher commented Feb 13, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

sschuberth commented Feb 28, 2024

fb33 commented Mar 7, 2024 • edited Loading

This comment has been minimized.

This comment has been minimized.

fb33 commented Mar 8, 2024

sschuberth commented Sep 20, 2024 • edited Loading

fb33 commented Sep 20, 2024

sschuberth commented Sep 20, 2024

fb33 commented Sep 20, 2024

sschuberth commented Oct 9, 2024

mnonnenmacher commented Oct 9, 2024

mnonnenmacher commented Feb 9, 2024 •

edited

Loading

mnonnenmacher commented Feb 13, 2024 •

edited

Loading

fb33 commented Mar 7, 2024 •

edited

Loading

sschuberth commented Sep 20, 2024 •

edited

Loading