[refine] prune outgroup

[jameshadfield]

Refactors augur refine to root trees before TreeTime is called, where possible. This allows us to easily add a --remove-outgroup flag which works only when the tree is rooted on a single strain, and the pruning of the root is performed before any temporal inference (if applicable). This approach is the one recommended by @rneher, @emmahodcroft and @huddlej in #340.

@joverlee521 I explored patching seasonal-flu to incorporate this however the structure there is a little complicated. Specifically, if the prune_reference rule is dropped in lieu of --remove-outgroup in the refine step then the reference sequences may/will be removed via the sanitize_trees step (which is before refine) as the references (often? always?) differ across segments.

Closes #340

[codecov]

Codecov Report

Attention: Patch coverage is 72.09302% with 12 lines in your changes missing coverage. Please review.

Project coverage is 73.25%. Comparing base (e6df775) to head (3d2ef06).
Report is 11 commits behind head on master.

Files with missing lines	Patch %	Lines
augur/refine.py	72.09%	8 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1751      +/-   ##
==========================================
+ Coverage   73.11%   73.25%   +0.14%     
==========================================
  Files          79       79              
  Lines        8350     8376      +26     
  Branches     1704     1706       +2     
==========================================
+ Hits         6105     6136      +31     
+ Misses       1958     1954       -4     
+ Partials      287      286       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[huddlej]

Thank you for making the ideas in #340 a reality here, @jameshadfield! I see what you mean about the complexity in seasonal-flu. You're right that the reference ids differ between segments right now, because we use sequence accession as the id.

Even if we renamed the reference ids to the strain name (which will be the same for all segments within a lineage), a better strategy might be to force sanitize_trees.py to keep the reference node through a new command line argument much like we prevent the earlier outlier flagging step from removing specific strains. The sanitize step will then produce outputs with the same strains across segments except for the reference strains which will get removed after rooting by refine. I would expect the tree.nwk after refine in this new workflow to be the same for downstream steps as the tree produced by the current workflow.

The other potential issue is that the flag_outliers script assumes that the input tree has already been rooted, but we wouldn't be rooting the tree until late at refine. This script has a --reroot flag, though, which we could add to the existing rule without reorganizing any other rules. Would it be helpful for me to mock up what the workflow would look like based on what I described above?

[jameshadfield]

I've shifted the seasonal-flu specific conversation to nextstrain/seasonal-flu#212

[j23414]

Testing with WNV

Awesome work! From a user's perspective, I tested this PR by attempting to root the WNV/lineage-1A tree using a lineage-1B outgroup and pruning it.

• View rooting problem

I included a lineage-1B outgroup strain (KX394399), which was later pruned out using the new --remove-outgroup refine flag. The result: rooting for the WNV/lineage-1A tree looks significantly improved! And it only required minimal workflow changes.

• View the updated tree

Thank you! No blockers from me from a user perspective.

[joverlee521]

Mainly reviewed for my own learning since I've been looking at the refine code recently with @kimandrews. Only left non-blocking comments.