Added an optimisation to spare not required shuffle of unpaired reads #2282

benraha · 2020-11-18T15:06:34Z

This PR adds an option to avoid shuffling the entire alignment dataset when transforming to FragmentDataset.

This adds a boolean parameter to the function toFragments() that, when specified, the FragmentDataset is created with one read per Fragment record.

This resolved issue #2281 .

…when they are unpaired

benraha · 2020-11-18T15:07:50Z

Jenkins, test this please

AmplabJenkins · 2020-11-18T15:39:59Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3131/

Build result: FAILURE

[...truncated 5 lines...]Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2282/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 1654f45 # timeout=10Checking out Revision 1654f45 (origin/pr/2282/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 1654f4582adc31c44bbf7eae908934548ba29afdFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,3.0.1,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb ? 2.7.5,2.12,2.4.7,ubuntuADAM-prb ? 2.7.5,2.12,3.0.1,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.7,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.12,2.4.7,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

heuermh · 2020-11-18T17:00:40Z

The Jenkins failure logs are a bit much to go through, this is the relevant part

...
[INFO] ------------< org.bdgenomics.adam:adam-python-spark2_2.11 >-------------
[INFO] Building ADAM_2.11: Python APIs 0.33.0-SNAPSHOT                    [8/9]
[INFO] --------------------------------[ jar ]---------------------------------
...
=================================== FAILURES ===================================
____________________ AlignmentDatasetTest.test_to_fragments ____________________

self = <bdgenomics.adam.test.alignmentDataset_test.AlignmentDatasetTest testMethod=test_to_fragments>

    def test_to_fragments(self):
    
        readsPath = self.resourceFile("unsorted.sam")
        ac = ADAMContext(self.ss)
    
        reads = ac.loadAlignments(readsPath)
    
>       fragments = reads.toFragments()

bdgenomics/adam/test/alignmentDataset_test.py:169: 
...
E                   py4j.protocol.Py4JError: An error occurred while calling o1728.toFragments. Trace:
E                   py4j.Py4JException: Method toFragments([]) does not exist
E                   	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:318)
E                   	at py4j.reflection.ReflectionEngine.getMethod(ReflectionEngine.java:326)
E                   	at py4j.Gateway.invoke(Gateway.java:274)
E                   	at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
E                   	at py4j.commands.CallCommand.execute(CallCommand.java:79)
E                   	at py4j.GatewayConnection.run(GatewayConnection.java:238)
E                   	at java.lang.Thread.run(Thread.java:748)

benraha · 2020-11-18T19:12:37Z

@heuermh Thanks! The tests should pass now, I'm ready to review and merge :)

AmplabJenkins · 2020-11-18T19:42:46Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3132/

Build result: FAILURE

[...truncated 5 lines...]Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2282/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains f142e4c # timeout=10Checking out Revision f142e4c (origin/pr/2282/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f f142e4c93b3a8733c1f790ac4c73fb6d138ee442First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,3.0.1,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb ? 2.7.5,2.12,2.4.7,ubuntuADAM-prb ? 2.7.5,2.12,3.0.1,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.7,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.12,2.4.7,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

benraha · 2020-11-19T08:27:17Z

Jenkins, test this please

AmplabJenkins · 2020-11-19T09:08:34Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3133/

Build result: FAILURE

[...truncated 5 lines...]Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2282/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains f142e4c # timeout=10Checking out Revision f142e4c (origin/pr/2282/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f f142e4c93b3a8733c1f790ac4c73fb6d138ee442First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,3.0.1,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb ? 2.7.5,2.12,2.4.7,ubuntuADAM-prb ? 2.7.5,2.12,3.0.1,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.7,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.12,2.4.7,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

benraha · 2020-11-19T18:30:35Z

Jenkins, test this please

AmplabJenkins · 2020-11-19T19:12:18Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3134/

Build result: FAILURE

[...truncated 5 lines...]Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2282/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains f142e4c # timeout=10Checking out Revision f142e4c (origin/pr/2282/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f f142e4c93b3a8733c1f790ac4c73fb6d138ee442First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,3.0.1,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb ? 2.7.5,2.12,2.4.7,ubuntuADAM-prb ? 2.7.5,2.12,3.0.1,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.7,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.12,2.4.7,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2020-11-19T21:06:21Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3135/

Build result: FAILURE

[...truncated 5 lines...]Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2282/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains bb2f078 # timeout=10Checking out Revision bb2f078 (origin/pr/2282/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f bb2f0780be008639963f08cff4fdddfaab8a37c4First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,3.0.1,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb ? 2.7.5,2.12,2.4.7,ubuntuADAM-prb ? 2.7.5,2.12,3.0.1,ubuntu completed with result SUCCESSADAM-prb ? 2.7.5,2.11,2.4.7,ubuntu completed with result SUCCESSADAM-prb ? 2.7.5,2.12,2.4.7,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

benraha · 2020-11-19T21:08:28Z

There seems to be some kind of issue unpacking the hadoop tar gz

https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3135/HADOOP_VERSION=2.7.5,SCALA_VERSION=2.12,SPARK_VERSION=2.4.7,label=ubuntu/console

heuermh · 2020-11-19T22:10:46Z

Sorry for all the trouble with Jenkins, I have seen something similar recently on another project

https://amplab.cs.berkeley.edu/jenkins/job/mango-prb/1002/HADOOP_VERSION=2.7.5,SCALAVER=2.12,SPARK_VERSION=2.4.7,label=amp-jenkins-staging-worker-02/console

That Apache mirror download link 404s, not sure what the actual issue is. Also odd that 2.7.7 doesn't match $HADOOP_VERSION . . . that will cause trouble when we add Hadoop 3.x to the build matrix (#2267).

heuermh · 2020-11-20T01:49:53Z

And a note from our admin staff:

i'm going to be upgrading jenkins to something more reasonable, and there will definitely be
some downtime as i get things sorted.

we should be back up and building by monday.

benraha · 2020-11-24T19:06:38Z

Jenkins, test this please

AmplabJenkins · 2020-11-24T20:01:31Z

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3136/

Build result: FAILURE

[...truncated 717 B...][WARNING] Reference path does not exist: /home/jenkins/gitcaches/adam.referenceFetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=30 > git --version # 'git version 1.7.1' > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=30 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse e6332ef^{commit} # timeout=30Checking out Revision e6332ef (detached) > git config core.sparsecheckout # timeout=30 > git checkout -f e6332ef # timeout=30Commit message: "code format fix"First time build. Skipping changelog.Triggering ADAM-prb » 2.7.5,2.12,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.12,3.0.1,ubuntuADAM-prb » 2.7.5,2.12,2.4.7,ubuntu completed with result FAILUREADAM-prb » 2.7.5,2.11,2.4.7,ubuntu completed with result SUCCESSADAM-prb » 2.7.5,2.12,3.0.1,ubuntu completed with result FAILURENotifying endpoint with url 'https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Setting status of e6332ef to FAILURE with url https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3136/ and message: 'Build finished. '

heuermh · 2020-11-24T20:08:05Z

re: Jenkins, might want to follow these threads

http://apache-spark-developers-list.1001551.n3.nabble.com/jenkins-downtime-tomorrow-evening-weekend-td30405.html

http://apache-spark-developers-list.1001551.n3.nabble.com/build-system-IMPORTANT-UPDATE-td30430.html

benraha · 2020-12-02T19:18:53Z

Jenkins, test this please

AmplabJenkins · 2020-12-02T20:14:18Z

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3137/

Build result: FAILURE

[...truncated 717 B...][WARNING] Reference path does not exist: /home/jenkins/gitcaches/adam.referenceFetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=30 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=30 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse e6332ef^{commit} # timeout=30Checking out Revision e6332ef (detached) > git config core.sparsecheckout # timeout=30 > git checkout -f e6332ef # timeout=30Commit message: "code format fix" > git rev-list --no-walk e6332ef # timeout=30Triggering ADAM-prb » 2.7.5,2.12,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.12,3.0.1,ubuntuADAM-prb » 2.7.5,2.12,2.4.7,ubuntu completed with result FAILUREADAM-prb » 2.7.5,2.11,2.4.7,ubuntu completed with result SUCCESSADAM-prb » 2.7.5,2.12,3.0.1,ubuntu completed with result SUCCESSNotifying endpoint with url 'https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Setting status of e6332ef to FAILURE with url https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3137/ and message: 'Build finished. '

heuermh · 2020-12-02T20:20:33Z

Re

The JAVA_HOME environment variable is not defined correctly
This environment variable is needed to run this program
NB: JAVA_HOME should point to a JDK not a JRE
Build step 'Execute shell' marked build as failure

This in Jenkins Build: Execute shell is apparently no longer correct

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export CONDA_BIN=/home/anaconda/bin/
export PATH=${JAVA_HOME}/bin/:${CONDA_BIN}:${PATH}

shaneknapp · 2020-12-03T00:27:17Z

test this please

shaneknapp · 2020-12-03T00:27:44Z

Re

The JAVA_HOME environment variable is not defined correctly
This environment variable is needed to run this program
NB: JAVA_HOME should point to a JDK not a JRE
Build step 'Execute shell' marked build as failure

This in Jenkins Build: Execute shell is apparently no longer correct

export JAVA_HOME=/usr/lib/jvm/java-8-oracle
export CONDA_BIN=/home/anaconda/bin/
export PATH=${JAVA_HOME}/bin/:${CONDA_BIN}:${PATH}

i fixed JAVA_HOME to point to /usr/java/latest (java 8)

AmplabJenkins · 2020-12-03T01:15:08Z

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3138/

Build result: FAILURE

[...truncated 713 B...][WARNING] Reference path does not exist: /home/jenkins/gitcaches/adam.referenceFetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=30 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=30 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=30Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse e6332ef^{commit} # timeout=30Checking out Revision e6332ef (detached) > git config core.sparsecheckout # timeout=30 > git checkout -f e6332ef # timeout=30Commit message: "code format fix" > git rev-list --no-walk e6332ef # timeout=30Triggering ADAM-prb » 2.7.5,2.12,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.12,3.0.1,ubuntuADAM-prb » 2.7.5,2.12,2.4.7,ubuntu completed with result FAILUREADAM-prb » 2.7.5,2.11,2.4.7,ubuntu completed with result SUCCESSADAM-prb » 2.7.5,2.12,3.0.1,ubuntu completed with result SUCCESSNotifying endpoint with url 'https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Setting status of e6332ef to FAILURE with url https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3138/ and message: 'Build finished. '

shaneknapp · 2020-12-03T02:59:24Z

test this please

shaneknapp · 2020-12-03T02:59:50Z

(i also updated the build config so that the main prb build runs on the workers, and not the primary node)

AmplabJenkins · 2020-12-03T04:15:38Z

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3139/

Build result: FAILURE

[...truncated 1.02 KB...]Using reference repository: /home/jenkins/gitcaches/adam.referenceFetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse e6332ef^{commit} # timeout=10Checking out Revision e6332ef (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f e6332ef # timeout=10Commit message: "code format fix" > git rev-list --no-walk e6332ef # timeout=10Triggering ADAM-prb » 2.7.5,2.12,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.11,2.4.7,ubuntuTriggering ADAM-prb » 2.7.5,2.12,3.0.1,ubuntuADAM-prb » 2.7.5,2.12,2.4.7,ubuntu completed with result FAILUREADAM-prb » 2.7.5,2.11,2.4.7,ubuntu completed with result SUCCESSADAM-prb » 2.7.5,2.12,3.0.1,ubuntu completed with result SUCCESSNotifying endpoint with url 'https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Setting status of e6332ef to FAILURE with url https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3139/ and message: 'Build finished. '

benraha · 2021-02-28T09:50:55Z

Jenkins, test this please

AmplabJenkins · 2021-02-28T10:41:43Z

Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3157/

Build result: FAILURE

[...truncated 905 B...]Wiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Using reference repository: /home/jenkins/gitcaches/adam.referenceFetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --force --progress -- https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse e6332ef^{commit} # timeout=10Checking out Revision e6332ef (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f e6332ef # timeout=10Commit message: "code format fix" > git rev-list --no-walk a895649 # timeout=10Triggering ADAM-prb » 3.2.1,2.12,3.0.2,ubuntuADAM-prb » 3.2.1,2.12,3.0.2,ubuntu completed with result FAILURENotifying endpoint with url 'https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Setting status of e6332ef to FAILURE with url https://amplab.cs.berkeley.edu/jenkins/job/ADAM-prb/3157/ and message: 'Build finished. '

Added an optimisation to not spare not required shuffle of the reads …

cc4a711

…when they are unpaired

added mergePairedReads as a parameter to Python API

d6a6678

code format fix

e6332ef

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added an optimisation to spare not required shuffle of unpaired reads #2282

Added an optimisation to spare not required shuffle of unpaired reads #2282

benraha commented Nov 18, 2020

benraha commented Nov 18, 2020

AmplabJenkins commented Nov 18, 2020

heuermh commented Nov 18, 2020

benraha commented Nov 18, 2020

AmplabJenkins commented Nov 18, 2020

benraha commented Nov 19, 2020

AmplabJenkins commented Nov 19, 2020

benraha commented Nov 19, 2020

AmplabJenkins commented Nov 19, 2020

AmplabJenkins commented Nov 19, 2020

benraha commented Nov 19, 2020

heuermh commented Nov 19, 2020

heuermh commented Nov 20, 2020

benraha commented Nov 24, 2020

AmplabJenkins commented Nov 24, 2020

heuermh commented Nov 24, 2020

benraha commented Dec 2, 2020

AmplabJenkins commented Dec 2, 2020

heuermh commented Dec 2, 2020

shaneknapp commented Dec 3, 2020

shaneknapp commented Dec 3, 2020

AmplabJenkins commented Dec 3, 2020

shaneknapp commented Dec 3, 2020

shaneknapp commented Dec 3, 2020

AmplabJenkins commented Dec 3, 2020

benraha commented Feb 28, 2021

AmplabJenkins commented Feb 28, 2021

Added an optimisation to spare not required shuffle of unpaired reads #2282

Are you sure you want to change the base?

Added an optimisation to spare not required shuffle of unpaired reads #2282

Conversation

benraha commented Nov 18, 2020

benraha commented Nov 18, 2020

AmplabJenkins commented Nov 18, 2020

Build result: FAILURE

heuermh commented Nov 18, 2020

benraha commented Nov 18, 2020

AmplabJenkins commented Nov 18, 2020

Build result: FAILURE

benraha commented Nov 19, 2020

AmplabJenkins commented Nov 19, 2020

Build result: FAILURE

benraha commented Nov 19, 2020

AmplabJenkins commented Nov 19, 2020

Build result: FAILURE

AmplabJenkins commented Nov 19, 2020

Build result: FAILURE

benraha commented Nov 19, 2020

heuermh commented Nov 19, 2020

heuermh commented Nov 20, 2020

benraha commented Nov 24, 2020

AmplabJenkins commented Nov 24, 2020

Build result: FAILURE

heuermh commented Nov 24, 2020

benraha commented Dec 2, 2020

AmplabJenkins commented Dec 2, 2020

Build result: FAILURE

heuermh commented Dec 2, 2020

shaneknapp commented Dec 3, 2020

shaneknapp commented Dec 3, 2020

AmplabJenkins commented Dec 3, 2020

Build result: FAILURE

shaneknapp commented Dec 3, 2020

shaneknapp commented Dec 3, 2020

AmplabJenkins commented Dec 3, 2020

Build result: FAILURE

benraha commented Feb 28, 2021

AmplabJenkins commented Feb 28, 2021

Build result: FAILURE