Launcher fails with Java 10 or later #953

ODiogoSilva · 2018-11-29T10:31:19Z

Bug report

Installing nextflow via conda on a fresh environment will install nextflow 18.10.1 and openjdk 10.0.2. However, executing nextflow with these versions yields the following:

$ nextflow -version 
CAPSULE EXCEPTION: Could not parse version line: openjdk version "10.0.2" 2018-07-17 (for stack trace, run with -Dcapsule.log=verbose)
USAGE: java <options> -jar ../miniconda3/envs/myenv/share/nextflow/framework/18.10.1/nextflow-18.10.1-one.jar

Actions:
  capsule.version - Prints the capsule and application versions.
  capsule.modes - Prints all available capsule modes.
  capsule.jvms - Prints a list of all JVM installations found.
  capsule.help - Prints this help message.
  capsule.tree - Prints the capsule's dependency tree.
  capsule.resolve - Downloads all un-cached dependencies.

Options:
  capsule.mode=<value> - Picks the capsule mode to run.
  capsule.reset - Resets the capsule cache before launching. The capsule to be re-extracted (if applicable), and other possibly cached files will be recreated.
  capsule.log=<value> (default: quiet) - Picks a log level. Must be one of none, quiet, verbose, or debug.
  capsule.java.home=<value> - Sets the location of the Java home (JVM installation directory) to use; If 'current' forces the use of the JVM that launched the capsule.
  capsule.java.cmd=<value> - Sets the path to the Java executable to use.
  capsule.jvm.args=<value> - Sets additional JVM arguments to use when running the application.
  capsule.local=<value> - Sets the path of the local Maven repository to use.
Unable to initialize nextflow environment

Is nextflow supposed to support java 10?

Environment

Nextflow version: 18.10.1
Java version: openjdk 10.0.2
Operating system: archlinux

Additional context

(Add any other context about the problem here)

The text was updated successfully, but these errors were encountered:

jenzopr · 2018-11-29T15:10:04Z

Confirmed. An easy solution would be to pin openjdk to <10 in bioconda.

pditommaso · 2018-11-30T07:53:41Z

That's weird we have integration tests for JDK 8 up to 11. What's the version string returned by your JDK?

$ java --version

jenzopr · 2018-11-30T08:17:22Z

For us, it returns

openjdk version "10.0.2" 2018-07-17
OpenJDK Runtime Environment Zulu10.3+5 (build 10.0.2+13)
OpenJDK 64-Bit Server VM Zulu10.3+5 (build 10.0.2+13, mixed mode)

Best,
Jens

pditommaso · 2018-11-30T08:22:32Z

This is mine

java version "10" 2018-03-20
Java(TM) SE Runtime Environment 18.3 (build 10+46)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10+46, mixed mode)

and

java version "11" 2018-09-25
Java(TM) SE Runtime Environment 18.9 (build 11+28)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11+28, mixed mode)

Apparently the Zulu distribution deviate from OpenJdk adding the minor version in that string causing this problem.

jenzopr · 2018-11-30T08:36:27Z

IMHO, it would definitely add to nextflows popularity if java versions delivered by large and state of the art software management tools (e.g. conda, in this case java comes from the conda-forge channel) would be seamlessly supported by nextflow. I guess there are numerous parsers for version numbers available in groovy (e.g. org.codehaus.groovy.grails.plugins.VersionComparator)?

pditommaso · 2018-11-30T08:41:20Z

The problem is not NF but Capsule package manager that seems to not recognise the version.

pditommaso · 2018-11-30T08:43:59Z

The Java spec for version number just for ref.

pditommaso · 2018-11-30T08:52:41Z

Actually this code.

jenzopr · 2018-11-30T09:04:14Z

Sorry, my bad about blaming nf then 😉
Is it worthwhile to file an issue upstream then? Unfortunately, I'm not fluent in groovy, so I can't contribute a minimal example..

jenzopr · 2018-11-30T09:06:40Z

I checked the regex pattern, it recognizes also minor versions..?

pditommaso · 2018-11-30T09:14:41Z

I've isolate the code

import java.io.IOException;
import java.util.Scanner;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

/**
 * @author Paolo Di Tommaso <paolo.ditommaso@gmail.com>
 */
public class Ver {

    public static void main(String... args) throws IOException {
        getActualJavaVersion();
    }

    private static final Pattern PAT_JAVA_VERSION_LINE = Pattern.compile(".*?\"(.+?)\"");

    private static void getActualJavaVersion() throws IOException  {
        System.out.println("Version property: " + System.getProperty("java.version"));
                
        Process process = new ProcessBuilder("java", "-version").redirectErrorStream(true).start();
        Scanner s = new Scanner(process.getInputStream()).useDelimiter("\n");
        String versionLine = s.hasNext() ? s.next() : "";
        System.out.println("Version string: " + versionLine);

        final Matcher m = PAT_JAVA_VERSION_LINE.matcher(versionLine);
        if (!m.matches()) {
            System.err.println("** Could not parse version line: " + versionLine);
        }
        else {
            final String version = m.group(1);
            System.out.println("Version detected: " + version);
        }
    }
}

It works for Java 8 and 9 and fails with 10 .. not sure to understand how test can pass on travis ? 😕

If you want to try it, save the above snippet, then use javac Ver.java && java -cp . Ver.

jenzopr · 2018-11-30T09:16:47Z

Thanks a lot! I can look into it and eventually propose a fix upstream.

pditommaso · 2018-11-30T11:04:10Z

Ok, looking better at this the regex .*?\"(.+?)\" fails because it assumes there's nothing else after the number, instead since java 10, it prints the build date java version "10" 2018-03-20.

However this does not explain why tests with OpenJDK are fine. I think this happens because before checking java -version, Capsule tries to infer the version number from the installation path:

https://github.com/puniverse/capsule/blob/master/capsule/src/main/java/Capsule.java#L4165-L4170

and

https://github.com/puniverse/capsule/blob/master/capsule/src/main/java/Capsule.java#L4189-L4216

I guess Azul is not recognised by the latter. Could you please check how is the installation path of your Azul JVM ?

pditommaso · 2018-11-30T11:48:22Z

Ahah, it looks conda mess up the Java install path, therefore it tries to use the java -version string, causing that issue.

$ which java
/Users/pditommaso/miniconda2/envs/nf-test/bin/java

jenzopr · 2018-11-30T12:46:39Z

Thanks for figuring this out! I'm fine with Capsule falling back to java -version and changing the pattern to ".*?\"(.+?)\".*?" solves the issue.

pditommaso · 2018-12-01T12:20:34Z

Requires puniverse/capsule#129 and puniverse/capsule#130.

dfornika · 2018-12-05T20:15:07Z

I've submitted this pull request to bioconda to pin openjdk=8 on the current nextflow-18.10.1 recipe as suggested above by @jenzopr:

bioconda/bioconda-recipes#12493

pditommaso · 2018-12-19T21:47:55Z

Patched in capsule fork 1.0.3.1 nextflow-io/capsule@df271ea

This commit fixes nextflow laucnher error when installing it whit Conda and using Java 10 or later. In a nutshell the Capsule, the package manager used by Nextflow is not able to parse the Java version number used since version 10. Therefore it tries to infer the version from the installation path, but Conda uses a custom format, therefore Capsule returns an error message. The problem is fixed by using a pacther version of Capsule forked from the original repo https://github.com/nextflow-io/capsule/tree/nextflow

jenzopr · 2018-12-20T09:00:09Z

Great! We'll give it a try.

pditommaso · 2018-12-20T09:00:56Z

Not yet released.

atongsa · 2021-07-20T14:34:11Z

Requires puniverse/capsule#129 and puniverse/capsule#130.

seems java faile to merge this pr;
i am using nextflow 21; and get the same error;
i solve it by mamba install openjdk"=8"

roryk · 2021-07-22T21:24:04Z

Just ran into this issue as well using bioconda installed nextflow on a google cloud linux box:

nextflow -h                                                                          (pythia_dataset) 
CAPSULE EXCEPTION: Could not parse version: 11.0.9.1-internal while processing attribute Min-Java-Version: 1.8.0 (for stack trace, run with -Dcapsule.log=verbose)
USAGE: java <options> -jar ../miniconda3/envs/pythia_dataset/share/nextflow/dist/21.04.0/nextflow-21.04.0-one.jar
Actions:
  capsule.version - Prints the capsule and application versions.
  capsule.modes - Prints all available capsule modes.
  capsule.jvms - Prints a list of all JVM installations found.
  capsule.help - Prints this help message.
  capsule.tree - Prints the capsule's dependency tree.
  capsule.resolve - Downloads all un-cached dependencies.
Options:
  capsule.mode=<value> - Picks the capsule mode to run.
  capsule.reset - Resets the capsule cache before launching. The capsule to be re-extracted (if applicable), and other possibly cached files will be recreated.
  capsule.log=<value> (default: quiet) - Picks a log level. Must be one of none, quiet, verbose, or debug.
  capsule.java.home=<value> - Sets the location of the Java home (JVM installation directory) to use; If 'current' forces the use of the JVM that launched the capsule.
  capsule.java.cmd=<value> - Sets the path to the Java executable to use.
  capsule.jvm.args=<value> - Sets additional JVM arguments to use when running the application.
  capsule.local=<value> - Sets the path of the local Maven repository to use.
Unable to initialize nextflow environment

openjdk 11.0.9.1-internal 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1-internal+0-adhoc..src)
OpenJDK 64-Bit Server VM (build 11.0.9.1-internal+0-adhoc..src, mixed mode)

not sure where the -internal is coming from, but that JDK is from the conda-forge one:

openjdk                   11.0.9.1             h5cc2fde_1    conda-forge

pditommaso · 2021-07-23T05:03:29Z

Stating this internal suffix identifies

a pre-release identifier. Typically ea, for an early-access release that's under active development and potentially unstable, or internal, for an internal developer build.

Make sure to use a stable Java release.

sstadick · 2021-07-28T14:38:52Z

It's not the -internal that is causing it to fail. It's this regex that is expecting the update to be _<number> not .number.

This broke for me going from 11.0.8-internal to 11.0.9.1-internal.

The current regex fails to parse `11.0.9.1-internal` due to the `.1` "update" group. This version is coming from openjdk conda forge. See [here](nextflow-io/nextflow#953 (comment)) for more details. This change allows for `_<update>` or `.<update>` in the version string.

pditommaso · 2021-07-28T16:09:50Z

You are right in this specific case, but depending the java version I've also seen failing for other reasons. For example with Java 9, crash because some Java cli options required by NF are not available with the internal version. Therefore we are not planning to support -internal distributions.

vincent-octo · 2021-10-20T06:32:32Z

I also had this error when wanting to install Nextflow via Bioconda.

I solved it by pinning openjdk to 11.0.8, which doesn't have the same problem as 11.0.9.1:
conda create -n myenv -c bioconda -c conda-forge openjdk=11.0.8 nextflow

colindaven · 2021-10-20T07:44:56Z

Interesting. I just solved it by using

conda deactivate

after manual download of nextflow "one.jar" to ~/.nextflow/framework/21.04.3/nextflow-21.04.3-one.jar

Edit: maybe a better option (for me) was to set the global environment variable
export NXF_JAVA_HOME=/mnt/ngsnfs/tools/jdk-10.0.1/
to a global, non-conda java.

Then I don't have to deactivate conda.

ArtemSokolov · 2022-03-21T15:05:02Z

One possible reason could be the number of tokens in the version number.
Users in our lab report having this issue with 11.0.14.1, but everything seems to work fine on my machine with 11.0.14.

sstadick · 2022-03-21T15:23:03Z

@ArtemSokolov, as Paolo mentioned, there are a few reasons why this issue pops up. You seem to have hit the same one as me, which is a regex that is expecting the "update" number as _<number>. See https://github.com/puniverse/capsule/pull/136/files

edit: it looks like capsule the project is dead, but @pditommaso has a fork of it? @pditommaso should I make my PR into your fork instead?

pditommaso · 2022-03-21T17:38:29Z

My point -internal version should not be used. There a plenty of Java stable distribution and it's dead easy to install with https://sdkman.io/

ArtemSokolov · 2022-03-21T17:53:44Z

I could be wrong, but I don't think that the problem-causing 11.0.14.1 is considered -internal.
Based on JEP 322, the fourth token is

$PATCH - The emergency patch-release counter, incremented only when it's necessary to produce an emergency release to fix a critical issue. (Using an additional element for this purpose minimizes disruption to both developers and users of in-flight update releases.)

which I believe is fair to consider as part of a stable release.

pditommaso · 2022-03-21T17:56:22Z

Then it should be supported.

safrabdh · 2022-06-23T09:54:40Z

Java Version Installed. Detected openjdk version "11.0.14.1". Please use Java 1.8.0

pditommaso added the triage/needs-information label Nov 30, 2018

jenzopr mentioned this issue Nov 30, 2018

PAT_JAVA_VERSION_LINE fails to recognize Zulu openjdk 10.0.2 puniverse/capsule#128

Open

pditommaso added priority and removed triage/needs-information labels Dec 1, 2018

dfornika mentioned this issue Dec 5, 2018

nextflow-18.10.1 fails to parse java version string bioconda/bioconda-recipes#12492

Closed

pditommaso added this to the v19.1.0 milestone Dec 19, 2018

pditommaso changed the title ~~Nextflow 18.10.1 does not initialize using OpenJDK 10.0.2~~ Launcher fails with Java 10 or later Dec 19, 2018

pditommaso closed this as completed Dec 19, 2018

sstadick mentioned this issue Jul 28, 2021

Fix PAT_JAVA_VERSION Regex puniverse/capsule#136

Open

ernstki mentioned this issue Nov 24, 2022

nextflow installation script yields "Could not parse version" error, OpenJDK 11.0.14.1 from MacPorts #3430

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Launcher fails with Java 10 or later #953

Launcher fails with Java 10 or later #953

ODiogoSilva commented Nov 29, 2018

jenzopr commented Nov 29, 2018

pditommaso commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018 •

edited

Loading

pditommaso commented Nov 30, 2018

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018

jenzopr commented Nov 30, 2018

pditommaso commented Dec 1, 2018 •

edited

Loading

dfornika commented Dec 5, 2018

pditommaso commented Dec 19, 2018

jenzopr commented Dec 20, 2018

pditommaso commented Dec 20, 2018 •

edited

Loading

atongsa commented Jul 20, 2021

roryk commented Jul 22, 2021

pditommaso commented Jul 23, 2021

sstadick commented Jul 28, 2021

pditommaso commented Jul 28, 2021

vincent-octo commented Oct 20, 2021

colindaven commented Oct 20, 2021 •

edited

Loading

ArtemSokolov commented Mar 21, 2022

sstadick commented Mar 21, 2022 •

edited

Loading

pditommaso commented Mar 21, 2022

ArtemSokolov commented Mar 21, 2022

pditommaso commented Mar 21, 2022

safrabdh commented Jun 23, 2022

Launcher fails with Java 10 or later #953

Launcher fails with Java 10 or later #953

Comments

ODiogoSilva commented Nov 29, 2018

Bug report

Environment

Additional context

jenzopr commented Nov 29, 2018

pditommaso commented Nov 30, 2018 • edited Loading

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018 • edited Loading

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018 • edited Loading

jenzopr commented Nov 30, 2018 • edited Loading

jenzopr commented Nov 30, 2018 • edited Loading

pditommaso commented Nov 30, 2018

jenzopr commented Nov 30, 2018

pditommaso commented Nov 30, 2018

pditommaso commented Nov 30, 2018

jenzopr commented Nov 30, 2018

pditommaso commented Dec 1, 2018 • edited Loading

dfornika commented Dec 5, 2018

pditommaso commented Dec 19, 2018

jenzopr commented Dec 20, 2018

pditommaso commented Dec 20, 2018 • edited Loading

atongsa commented Jul 20, 2021

roryk commented Jul 22, 2021

pditommaso commented Jul 23, 2021

sstadick commented Jul 28, 2021

pditommaso commented Jul 28, 2021

vincent-octo commented Oct 20, 2021

colindaven commented Oct 20, 2021 • edited Loading

ArtemSokolov commented Mar 21, 2022

sstadick commented Mar 21, 2022 • edited Loading

pditommaso commented Mar 21, 2022

ArtemSokolov commented Mar 21, 2022

pditommaso commented Mar 21, 2022

safrabdh commented Jun 23, 2022

pditommaso commented Nov 30, 2018 •

edited

Loading

pditommaso commented Nov 30, 2018 •

edited

Loading

pditommaso commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018 •

edited

Loading

jenzopr commented Nov 30, 2018 •

edited

Loading

pditommaso commented Dec 1, 2018 •

edited

Loading

pditommaso commented Dec 20, 2018 •

edited

Loading

colindaven commented Oct 20, 2021 •

edited

Loading

sstadick commented Mar 21, 2022 •

edited

Loading