[ADAM-384] Adds import from FASTQ. #385
Conversation
| * quality string? | ||
| * | ||
| * For now I'm going to assume single-line sequences. This works for our sequencing | ||
| * application. We'll see if someone complains in other applications. |
There was a problem hiding this comment.
The possibility of multi-line FASTQ sequences is a well known bug in the format definition -- for precisely the reason you point out here. +1 on your call to "ignore, and revisit if someone complains."
There was a problem hiding this comment.
Agreed, bug/"feature". Alas, the curse of text based file formats.
Anyways, we'll need to fix that (sooner rather than later); I chose to not tackle it now, because Hadoop-BAM hadn't tackled it either.
|
See my comments inline; this PR also needs a rebase off the latest master, @fnothaft. Otherwise, it's looking pretty good to me! |
|
Changes made and code is rebased. Thanks for the review, @tdanford ! |
|
Test PASSed. |
[ADAM-384] Adds import from FASTQ.
|
Thanks, Frank! |
Resolves #384. Adds:
Load from single ended FASTQ and interleaved FASTQ are handled seamlessly by the ADAMContext.adamLoad method. Since paired ended (but non-interleaved) data requires two file paths, I haven't added it into adamLoad; thus, it sits on its own.
I wrote our own FASTQ input format instead of using the one in Hadoop-BAM; theirs is only compatible with Hadoop 1, performs unnecessary parsing, and doesn't seem to pick splits correctly anyways. The SingleFastqInputFormat is almost a direct copy of the InterleavedFastqInputFormat, which we've tested pretty well on clusters.