[ADAM-946] Fixes to FlagStat for Samtools concordance issue #954
Conversation
|
Can one of the admins verify this patch? |
|
Jenkins, add to whitelist. |
|
Stellar! Thanks for sorting this out, @jpdna! Interesting to see that it is the "unmapped, yet have alignment info" reads that are causing the trouble. I've always been unsure as to what is the correct way to handle them. |
|
Test FAILed. Build result: FAILUREGitHub pull request #954 of commit 6a497b9 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/954/merge^{commit} # timeout=10 > git branch -a --contains 6792655 # timeout=10 > git rev-parse remotes/origin/pr/954/merge^{commit} # timeout=10Checking out Revision 6792655 (origin/pr/954/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 6792655a74f389324d18134be2bca4b8a5f773bfFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.4.1,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'Test FAILed. |
|
oops, I see now this doesn't pass the tests. |
I'd need to read the source to be 100%, but I think |
| // the statistics output by other programs, specifically Samtools Flagstat | ||
|
|
||
| if (samRecord.getReadUnmappedFlag) { | ||
| builder.setReadMapped(false) |
There was a problem hiding this comment.
This can be simplified to...builder.setReadMapped(!samRecord.getReadUnmappedFlag) .. without needing a conditional.
| @@ -249,7 +249,7 @@ class AlignmentRecordConverterSuite extends FunSuite { | |||
| .split('\n') | |||
|
|
|||
| assert(!secondRecord.getReadMapped) | |||
| assert(!secondRecord.getReadNegativeStrand) | |||
There was a problem hiding this comment.
Given the input data, this seems likely a bug in the test that was previously covered up by fact that for unmapped reads the code previously did not set this flag even if present in input.
|
Test PASSed. |
|
I've pushed changes to address flag setting code simplification @massie suggests above and also modified tests in a way that the code now passes the tests. The tests modifications seem to remove a bug or unnecessary requirements, but should be reviewed by others to make sure no downstream assumptions are violated. |
| @@ -78,7 +78,7 @@ private[adam] object DecadentRead extends Logging with Serializable { | |||
| @deprecated("Use RichAlignmentRecord wherever possible in new development.", since = "0.18.0") | |||
| private[adam] class DecadentRead(val record: RichAlignmentRecord) extends Logging { | |||
| // Can't be a primary alignment unless it has been aligned | |||
| require(!record.getPrimaryAlignment || record.getReadMapped, "Unaligned read can't be a primary alignment") | |||
| //require(!record.getPrimaryAlignment || record.getReadMapped, "Unaligned read can't be a primary alignment") | |||
There was a problem hiding this comment.
You might consider logging a warning message or dropping this line altogether instead of commenting out the code.
There was a problem hiding this comment.
You might consider logging a warning message or dropping this line altogether instead of commenting out the code.
Done. - removed the require line that had been commented-out
|
Test PASSed. |
| // are unclear when the read is not mapped or reference is not defined, | ||
| // it is nonetheless favorable to set these flags in the ADAM file | ||
| // in same way as they appear in the input BAM inorder to match exactly | ||
| // the statistics output by other programs, specifically Samtools Flagstat |
|
LGTM |
heuermh
changed the title
|
Fixes #946 |
|
Thank you, @jpdna. Merged manually. |
These changes result in perfect concordance between ADAM and Samtools flagstat output stats, addressing corner cases around secondary, supplemental, and unmapped reads as described in #946