[ADAM-1695] Check for illegal genotype index after splitting multi-allelic variants. #1725

heuermh · 2017-09-14T18:08:31Z

fnothaft · 2017-09-14T18:10:53Z

OOC, do you know what the license on the GIAB data is? We should ensure that it is OK to check in an excerpt of the GIAB truth file.

fnothaft

LGTM, just dropped a question on the code, and a question WRT importing the GIAB file. Thanks @heuermh!

fnothaft · 2017-09-14T18:15:41Z

adam-core/src/main/scala/org/bdgenomics/adam/converters/VariantContextConverter.scala

@@ -787,7 +787,7 @@ class VariantContextConverter(
                                             gIndices: Array[Int]): Genotype.Builder = {

    // AD is an array type field
-    if (g.hasAD) {
+    if (g.hasAD && gIdx < g.getAD.size) {


Does this only affect the AD field? I would've expected it to broadly impact fields that have the same array count as AD? Do we already check those?

Yes, that file also has an ADALL VCF FORMAT Number=R Type=Integer genotype field that causes ArrayIndexOutOfBoundsExceptions, thus the other change at line 1289.

I'll add additional assertions to the test case to demonstrate this fix.

Oh, hah! I'd missed that change earlier. Thanks for clarifying!

AmplabJenkins · 2017-09-14T18:34:06Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2373/

Build result: FAILURE

[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1725/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 1b8213b463b23656995b7d4aa0d80b853419944a # timeout=10Checking out Revision 1b8213b463b23656995b7d4aa0d80b853419944a (origin/pr/1725/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1b8213b463b23656995b7d4aa0d80b853419944aFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.0,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.0,centosADAM-prb ? 2.6.2,2.11,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.11,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.11,2.2.0,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.10,1.6.3,centos completed with result FAILUREADAM-prb ? 2.6.2,2.10,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.6.2,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

heuermh · 2017-09-14T19:31:02Z

OOC, do you know what the license on the GIAB data is? We should ensure that it is OK to check in an excerpt of the GIAB truth file.

"License: There are no restrictions on the use of this data."

https://aws.amazon.com/public-datasets/giab

"[P]lease cite http://www.nature.com/nbt/journal/v32/n3/full/nbt.2835.html (doi:10.1038/nbt.2835) and http://www.nature.com/articles/sdata201625 (doi:10.1038/sdata.2016.25) when using these calls."

ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/NA12878_HG001/latest/README_NISTv3.3.2.txt

I can add an accompanying HG...vcf.README to the adam-core/src/test/resources directory.

fnothaft · 2017-09-14T19:36:11Z

I can add an accompanying HG...vcf.README to the adam-core/src/test/resources directory.

+1

fnothaft · 2017-09-14T19:39:39Z

adam-core/src/main/scala/org/bdgenomics/adam/converters/VariantContextConverter.scala

@@ -787,7 +787,7 @@ class VariantContextConverter(
                                             gIndices: Array[Int]): Genotype.Builder = {

    // AD is an array type field
-    if (g.hasAD) {
+    if (g.hasAD && gIdx < g.getAD.size) {


Oh, hah! I'd missed that change earlier. Thanks for clarifying!

AmplabJenkins · 2017-09-14T20:51:15Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2376/

Build result: FAILURE

[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1725/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 8d26c62 # timeout=10Checking out Revision 8d26c62 (origin/pr/1725/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 8d26c627d44a9a0e692ff6767e6759902dd84523First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.0,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.0,centosADAM-prb ? 2.6.2,2.11,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.11,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.11,2.2.0,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.10,1.6.3,centos completed with result FAILUREADAM-prb ? 2.6.2,2.10,1.6.3,centos completed with result SUCCESSADAM-prb ? 2.6.2,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2017-09-14T21:39:32Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2377/
Test PASSed.

heuermh · 2017-09-27T18:43:45Z

Rebased, ready for review @devin-petersohn

AmplabJenkins · 2017-09-27T18:49:26Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2403/

Build result: FAILURE

[...truncated 15 lines...] > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1725/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains c00a9decf806b3cfe01e243f49ebd8a5f0872594 # timeout=10Checking out Revision c00a9decf806b3cfe01e243f49ebd8a5f0872594 (origin/pr/1725/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f c00a9decf806b3cfe01e243f49ebd8a5f0872594First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,1.6.3,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,1.6.3,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.0,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.0,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.0,centosADAM-prb ? 2.6.2,2.11,1.6.3,centos completed with result FAILUREADAM-prb ? 2.7.3,2.11,1.6.3,centos completed with result FAILUREADAM-prb ? 2.7.3,2.11,2.2.0,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,1.6.3,centos completed with result FAILUREADAM-prb ? 2.6.2,2.10,1.6.3,centos completed with result FAILUREADAM-prb ? 2.6.2,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.7.3,2.10,2.2.0,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.0,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2017-09-27T19:20:54Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2404/
Test PASSed.

devin-petersohn · 2017-09-27T20:22:25Z

Merged, thanks @heuermh!

fnothaft approved these changes Sep 14, 2017

View reviewed changes

heuermh force-pushed the issue-1695 branch from f7dcfc5 to 49e91ce Compare September 14, 2017 21:16

Check for illegal genotype index after splitting multi-allelic variants.

7021ecd

heuermh force-pushed the issue-1695 branch from 49e91ce to 7021ecd Compare September 27, 2017 18:43

fix merge issue

4de54b9

devin-petersohn approved these changes Sep 27, 2017

View reviewed changes

devin-petersohn merged commit 1302e07 into bigdatagenomics:master Sep 27, 2017

heuermh deleted the issue-1695 branch September 27, 2017 21:19

heuermh added this to the 0.23.0 milestone Oct 4, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ADAM-1695] Check for illegal genotype index after splitting multi-allelic variants. #1725

[ADAM-1695] Check for illegal genotype index after splitting multi-allelic variants. #1725

heuermh commented Sep 14, 2017

fnothaft commented Sep 14, 2017

fnothaft left a comment

fnothaft Sep 14, 2017

heuermh Sep 14, 2017

fnothaft Sep 14, 2017

AmplabJenkins commented Sep 14, 2017

heuermh commented Sep 14, 2017

fnothaft commented Sep 14, 2017

fnothaft Sep 14, 2017

AmplabJenkins commented Sep 14, 2017

AmplabJenkins commented Sep 14, 2017

heuermh commented Sep 27, 2017

AmplabJenkins commented Sep 27, 2017

AmplabJenkins commented Sep 27, 2017

devin-petersohn commented Sep 27, 2017

[ADAM-1695] Check for illegal genotype index after splitting multi-allelic variants. #1725

[ADAM-1695] Check for illegal genotype index after splitting multi-allelic variants. #1725

Conversation

heuermh commented Sep 14, 2017

fnothaft commented Sep 14, 2017

fnothaft left a comment

Choose a reason for hiding this comment

fnothaft Sep 14, 2017

Choose a reason for hiding this comment

heuermh Sep 14, 2017

Choose a reason for hiding this comment

fnothaft Sep 14, 2017

Choose a reason for hiding this comment

AmplabJenkins commented Sep 14, 2017

Build result: FAILURE

heuermh commented Sep 14, 2017

fnothaft commented Sep 14, 2017

fnothaft Sep 14, 2017

Choose a reason for hiding this comment

AmplabJenkins commented Sep 14, 2017

Build result: FAILURE

AmplabJenkins commented Sep 14, 2017

heuermh commented Sep 27, 2017

AmplabJenkins commented Sep 27, 2017

Build result: FAILURE

AmplabJenkins commented Sep 27, 2017

devin-petersohn commented Sep 27, 2017