Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. #4941

LeeTL1220 · 2018-06-22T19:11:52Z

MAF is the output of Funcotator in M2 WDL. Closes When Funcotator is run in the mutect2.wdl, it should output MAF, not VCF (by default) #4935
Updated mutect2.wdl manually tested locally and manually tested in FireCloud.
Updated mutect2_nio.wdl manually tested in FireCloud.
Updated automatic Cromwell WDL tests. Closes Enable running of Funcotator in the M2 WDL tests. #4807
Empty MAFs will be devoid of variants, not a file of 0 bytes. Closes MAFOutputRenderer produces an empty file when the input VCF has no variants to render. #4937
Fixed issue where multiple transcripts could be selected in edge cases, even when CANONICAL or BEST_EFFECT was selected. Closes Funcotator can select more than one transcript, even when CANONICAL or BEST_EFFECT is chosen. #4952

codecov-io · 2018-06-22T20:23:50Z

Codecov Report

Merging #4941 into master will increase coverage by 0.002%.
The diff coverage is 94.286%.

@@              Coverage Diff               @@
##              master    #4941       +/-   ##
==============================================
+ Coverage     80.468%   80.47%   +0.002%     
- Complexity     17852    17856        +4     
==============================================
  Files           1093     1093               
  Lines          64295    64302        +7     
  Branches       10378    10377        -1     
==============================================
+ Hits           51737    51744        +7     
  Misses          8504     8504               
  Partials        4054     4054

Impacted Files	Coverage Δ	Complexity Δ
...cotator/mafOutput/CustomMafFuncotationCreator.java	`90% <100%> (ø)`	`17 <1> (ø)`	⬇️
.../tools/funcotator/mafOutput/MafOutputRenderer.java	`95.908% <100%> (+0.042%)`	`59 <9> (+2)`	⬆️
...dataSources/gencode/GencodeFuncotationFactory.java	`83.383% <80%> (+0.075%)`	`162 <6> (+2)`	⬆️

…tputs have a header that includes custom fields.

LeeTL1220 · 2018-07-05T20:43:11Z

@davidbenjamin
This will make the WDL default to producing a MAF from Funcotator instead of a VCF. There is no flag to switch between the two, so if you know of people that still want VCF output, please speak up now...

Can you review the WDL and autotest-WDL changes? This has been tested in FireCloud and looks good (minus an issue that I have already filed), though I had to manually review. The Method configuration in FireCloud still uses the GATK jar override for this. Just in case you wanted to run it. Otherwise, we should blank out that parameter.

As a reminder, I tested:

mutect2.wdl: manually on local backend and FireCloud
mutect2_nio.wdl: manually on FireCloud

Please review both WDL files.

@jonn-smith Could you review the rest? I.e. the bug fixes.

Apologies that I did not split these into two PRs.

LeeTL1220 · 2018-07-05T20:43:39Z

@davidbenjamin @jonn-smith The test failure is a false alarm. Please review as if the tests were passing.

davidbenjamin

I'm satisfied.

davidbenjamin · 2018-07-12T01:56:05Z

scripts/mutect2_wdl/mutect2.wdl

+     String annotation_over_arg = if defined(annotation_overrides) then " --annotation-override " else ""
+     String filter_funcotations_args = if (filter_funcotations) then " --remove-filtered-variants " else ""
+     String interval_list_arg = if defined(interval_list) then " -L " else ""
+     String extra_args_arg = select_first([extra_args, ""])


This is the same as just $extra_args

(in mutect2_nio.wdl as well)

davidbenjamin · 2018-07-12T01:56:41Z

scripts/mutect2_wdl/mutect2.wdl

+     String annotation_def_arg = if defined(annotation_defaults) then " --annotation-default " else ""
+     String annotation_over_arg = if defined(annotation_overrides) then " --annotation-override " else ""
+     String filter_funcotations_args = if (filter_funcotations) then " --remove-filtered-variants " else ""
+     String interval_list_arg = if defined(interval_list) then " -L " else ""


This accomplishes the same as just ${"-L " intervals}

${"-L " + interval_list} Done

(in mutect2_nio.wdl as well)

davidbenjamin · 2018-07-12T01:57:25Z

scripts/mutect2_wdl/mutect2.wdl

+     Boolean use_ssd = false
+
+     # This should be updated when a new version of the data sources is released
+     # TODO: Make this dynamically chosen in the command.


What about this TODO?

This was out of date and has been eliminated in favor of a reasonable default. Also, we will probably remove the FTP download at some point, since it often breaks.

Done

davidbenjamin · 2018-07-12T02:08:14Z

...va/org/broadinstitute/hellbender/tools/funcotator/mafOutput/CustomMafFuncotationCreator.java

@@ -27,7 +27,7 @@
 */
 public class CustomMafFuncotationCreator {

-    final static List<String> COUNT_FIELD_NAMES = Arrays.asList(
+    final public static List<String> COUNT_FIELD_NAMES = Arrays.asList(


public final

davidbenjamin · 2018-07-12T02:09:00Z

src/main/java/org/broadinstitute/hellbender/tools/funcotator/mafOutput/MafOutputRenderer.java

@@ -220,6 +220,10 @@ public MafOutputRenderer(final Path outputFilePath,

    @Override
    public void close() {
+        if (!hasWrittenHeader) {
+            final LinkedHashMap<String, String> mafCompliantOutputMap = createMafCompliantOutputMap(Allele.create("AT"), Collections.emptyList());


What is "AT"?

Put in some clarifying comments.

Done.

LeeTL1220 added the Funcotator label Jun 22, 2018

LeeTL1220 force-pushed the ll_4935_Funcotator_MAF_WDL branch from d4a9471 to 40341bb Compare June 22, 2018 21:03

LeeTL1220 added 11 commits July 3, 2018 09:22

First cut. Runs but found an issue.

028316a

Fixes #4937

65793a6

Updates for turning on Funcotator in the M2 autotesting of WDL.

c4922ff

Wiring up the WDL test.

6295f6a

Adding a little more to the integration test to make sure that MAF ou…

f8a980b

…tputs have a header that includes custom fields.

Doc updates.

6f63b77

Doc updates.

33dcbad

- Fixes #4952

129a968

Non-NIO WDL works in FireCloud. NIO, not so much.

dab2f3d

Getting extra funcotator args into the WDL.

18b9e91

Adding more args into Funcotator M2 WDL.

5fa68f9

LeeTL1220 force-pushed the ll_4935_Funcotator_MAF_WDL branch from 9150721 to 5fa68f9 Compare July 3, 2018 13:22

LeeTL1220 added 3 commits July 3, 2018 09:47

Addressing rebase issues.

9a4fa40

Addressing funcotator size estimation.

2002123

Fixed broken test.

d4c5ef5

LeeTL1220 changed the title ~~Making MAF become the output of Funcotator in M2 WDL. (Do not merge yet)~~ Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. (Do not merge yet) Jul 5, 2018

LeeTL1220 added the PRIORITY_HIGH label Jul 5, 2018

LeeTL1220 requested a review from davidbenjamin July 5, 2018 20:35

LeeTL1220 changed the title ~~Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. (Do not merge yet)~~ Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. Jul 5, 2018

davidbenjamin self-assigned this Jul 12, 2018

davidbenjamin approved these changes Jul 12, 2018

View reviewed changes

Answering PRs.

9df8d60

LeeTL1220 merged commit 1977c53 into master Jul 16, 2018

LeeTL1220 deleted the ll_4935_Funcotator_MAF_WDL branch July 16, 2018 15:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. #4941

Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. #4941

LeeTL1220 commented Jun 22, 2018 •

edited

Loading

codecov-io commented Jun 22, 2018 •

edited

Loading

LeeTL1220 commented Jul 5, 2018

LeeTL1220 commented Jul 5, 2018

davidbenjamin left a comment

davidbenjamin Jul 12, 2018

LeeTL1220 Jul 16, 2018 •

edited

Loading

LeeTL1220 Jul 16, 2018

davidbenjamin Jul 12, 2018

LeeTL1220 Jul 16, 2018

LeeTL1220 Jul 16, 2018

davidbenjamin Jul 12, 2018

LeeTL1220 Jul 16, 2018

davidbenjamin Jul 12, 2018

LeeTL1220 Jul 16, 2018

davidbenjamin Jul 12, 2018

LeeTL1220 Jul 16, 2018

Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. #4941

Making MAF become the output of Funcotator in M2 WDL and multiple transcript fix. #4941

Conversation

LeeTL1220 commented Jun 22, 2018 • edited Loading

codecov-io commented Jun 22, 2018 • edited Loading

Codecov Report

LeeTL1220 commented Jul 5, 2018

LeeTL1220 commented Jul 5, 2018

davidbenjamin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeeTL1220 Jul 16, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

LeeTL1220 commented Jun 22, 2018 •

edited

Loading

codecov-io commented Jun 22, 2018 •

edited

Loading

LeeTL1220 Jul 16, 2018 •

edited

Loading