Merge pull request #580 from nf-core/dev

Release 2.2.0
nf-core · Sep 13, 2024 · fa61a65 · fa61a65
2 parents c7a6f38 + fb77d3f
commit fa61a65
Show file tree

Hide file tree

Showing 72 changed files with 2,025 additions and 1,139 deletions.
diff --git a/.github/workflows/download_pipeline.yml b/.github/workflows/download_pipeline.yml
@@ -69,11 +69,3 @@ jobs:
 
       - name: Inspect download
         run: tree ./${{ env.REPOTITLE_LOWERCASE }}
-
-      - name: Run the downloaded pipeline (stub)
-        id: stub_run_pipeline
-        continue-on-error: true
-        env:
-          NXF_SINGULARITY_CACHEDIR: ./
-          NXF_SINGULARITY_HOME_MOUNT: true
-        run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -3,6 +3,76 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 2.2.0 - Dogmatix [2024-09-13]
+
+### `Added`
+
+- A new parameter `mt_aligner` to control which aligner is used to align reads to mitochondria [#600](https://github.com/nf-core/raredisease/pull/600)
+- A new parameter `par_bed` to pass a PAR bed files to deepvariant [#598](https://github.com/nf-core/raredisease/pull/598)
+- A new functionality to pass gzipped resources to vcfanno_extra_resources [#589](https://github.com/nf-core/raredisease/pull/589)
+- A new parameter `vcfanno_extra_resources` to pass an extra resource to vcfanno [#588](https://github.com/nf-core/raredisease/pull/588)
+- A new parameter `scatter_count` to control how many interval files are created from a genome (used to parallelize annotations) [#585](https://github.com/nf-core/raredisease/pull/585)
+- Print warning messages if user intends to perform ranking when there are no affected samples [#579](https://github.com/nf-core/raredisease/pull/579)
+- Two new parameters `skip_repeat_annotation` and `skip_repeat_calling` to skip calling and annotation of repeat expansions [#574](https://github.com/nf-core/raredisease/pull/574)
+- A new parameter `skip_smncopynumbercaller` to skip smncopynumbercaller module [#574](https://github.com/nf-core/raredisease/pull/574)
+- A new parameter `skip_sv_calling` to skip sv calling workflow [#572](https://github.com/nf-core/raredisease/pull/572)
+- Two new parameters `skip_snv_calling` and `skip_repeat_analysis` to skip snv calling and repeat analysis respectively [#571](https://github.com/nf-core/raredisease/pull/571)
+- Two new parameters `mbuffer_mem` and `samtools_sort_threads` to control resources given to mbuffer and samtools sort in the bwameme module [#570](https://github.com/nf-core/raredisease/pull/570)
+
+### `Changed`
+
+- Update default vep container from v110-v112 [#609](https://github.com/nf-core/raredisease/pull/609)
+- Default index for vcfanno extra annotation files from tbi to csi [#606](https://github.com/nf-core/raredisease/pull/606)
+- Updated the model for Sentieon DNAScope to v1.1 [#601](https://github.com/nf-core/raredisease/pull/601)
+- bwameme can no longer be used to align mitochondrial reads [#600](https://github.com/nf-core/raredisease/pull/600)
+- Males' X and Y chromosomes will be treated as haploids during variant calling by deepvariant [#598](https://github.com/nf-core/raredisease/pull/598)
+- Acceptable type for lane field in the samplesheet from number to string [#597](https://github.com/nf-core/raredisease/pull/597)
+- Allow `0` as a valid value for `sex` in the samplesheet [#595](https://github.com/nf-core/raredisease/pull/595)
+- Updated deepvariant to version 1.6.1 [#587](https://github.com/nf-core/raredisease/pull/587)
+- Parallelized vcfanno [#585](https://github.com/nf-core/raredisease/pull/585)
+- Skip ROH calling with bcftools if there are no affected samples [#579](https://github.com/nf-core/raredisease/pull/579)
+- Refactored tool citation list [#577](https://github.com/nf-core/raredisease/pull/577)
+- Removed `skip_repeat_analysis` added in #571 [#574](https://github.com/nf-core/raredisease/pull/574)
+- Remove several skip parameters that had been included in the pipeline to avoid failed CI tests (see parameters table below) [#574](https://github.com/nf-core/raredisease/pull/574)
+- `readcount_intervals` parameter is now mandatory for running germlinecnvcaller. [#570](https://github.com/nf-core/raredisease/pull/570)
+- Turn off CNVnator, TIDDIT, SMNCopyNumberCaller, Gens, and Vcf2cytosure for targeted analysis [#573](https://github.com/nf-core/raredisease/pull/573)
+
+### `Fixed`
+
+- Issues that cropped up when `aligner` and `mt_aligner` were different [#605](https://github.com/nf-core/raredisease/pull/605)
+- Update docs to show 'vep_plugin_files' as a mandatory parameter for SNV annotation [#594](https://github.com/nf-core/raredisease/pull/594)
+- Error in SVDB merge when only a single SV caller is run [#586](https://github.com/nf-core/raredisease/pull/586)
+- Errors due to misplaced version statements [#578](https://github.com/nf-core/raredisease/pull/578)
+- Stub crashes due to peddy reported in [#566](https://github.com/nf-core/raredisease/issues/566) [#576](https://github.com/nf-core/raredisease/pull/576]
+- Docker manifest error from gnu-wget container [#570](https://github.com/nf-core/raredisease/pull/570)
+- Citations for bwameme [#563](https://github.com/nf-core/raredisease/pull/563)
+
+### Parameters
+
+| Old parameter   | New parameter            |
+| --------------- | ------------------------ |
+|                 | mbuffer_mem              |
+|                 | mt_aligner               |
+|                 | samtools_sort_threads    |
+|                 | skip_repeat_calling      |
+|                 | skip_snv_calling         |
+|                 | skip_sv_calling          |
+| skip_eklipse    |                          |
+| skip_fastqc     |                          |
+| skip_haplocheck |                          |
+| skip_qualimap   |                          |
+|                 | skip_smncopynumbercaller |
+|                 | skip_repeat_annotation   |
+|                 | scatter_count            |
+|                 | vcfanno_extra_resources  |
+
+### Tool updates
+
+| Tool        | Old version | New version |
+| ----------- | ----------- | ----------- |
+| Deepvariant | 1.5.0       | 1.6.1       |
+| ensemblvep  | 110         | 112         |
+
 ## 2.1.0 - Obelix [2024-05-29]
 
 ### `Added`

diff --git a/CITATIONS.md b/CITATIONS.md
@@ -22,6 +22,10 @@
 
   > Vasimuddin Md, Misra S, Li H, Aluru S. Efficient Architecture-Aware Acceleration of BWA-MEM for Multicore Systems. In: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE; 2019:314-324. doi:10.1109/IPDPS.2019.00041
 
+- [BWA-MEME](https://academic.oup.com/bioinformatics/article/38/9/2404/6543607)
+
+  > Jung Y, Han D. BWA-MEME: BWA-MEM emulated with a machine learning approach. Bioinformatics. 2022;38(9):2404-2413. doi:10.1093/bioinformatics/btac137
+
 - [CADD<sup>1</sup>](https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-021-00835-9)<sup>,</sup> [<sup>2</sup>](https://academic.oup.com/nar/article/47/D1/D886/5146191)
 
   > Rentzsch P, Schubach M, Shendure J, Kircher M. CADD-Splice—improving genome-wide variant effect prediction using deep learning-derived splice scores. Genome Med. 2021;13(1):31. doi:10.1186/s13073-021-00835-9

diff --git a/assets/multiqc_config.yml b/assets/multiqc_config.yml
@@ -3,9 +3,9 @@ custom_logo_url: https://github.com/nf-core/raredisease/
 custom_logo_title: "nf-core/raredisease"
 
 report_comment: >
-  This report has been generated by the <a href="https://github.com/nf-core/raredisease/releases/tag/2.1.0" target="_blank">nf-core/raredisease</a>
+  This report has been generated by the <a href="https://github.com/nf-core/raredisease/releases/tag/2.2.0" target="_blank">nf-core/raredisease</a>
   analysis pipeline. For information about how to interpret these results, please see the
-  <a href="https://nf-co.re/raredisease/2.1.0/docs/output" target="_blank">documentation</a>.
+  <a href="https://nf-co.re/raredisease/2.2.0/docs/output" target="_blank">documentation</a>.
 report_section_order:
   "nf-core-raredisease-methods-description":
     order: -1000

diff --git a/assets/schema_input.json b/assets/schema_input.json
@@ -15,8 +15,9 @@
                 "errorMessage": "Sample name must be provided and cannot contain spaces"
             },
             "lane": {
-                "type": "number",
-                "meta": ["lane"]
+                "type": "string",
+                "meta": ["lane"],
+                "pattern": "^\\S+$"
             },
             "fastq_1": {
                 "type": "string",
@@ -42,7 +43,7 @@
             "sex": {
                 "type": "string",
                 "meta": ["sex"],
-                "enum": ["1", "2", "other"],
+                "enum": ["1", "2", "0", "other"],
                 "errorMessage": "Sex must be provided and cannot contain spaces"
             },
             "phenotype": {

diff --git a/conf/modules/align_MT.config b/conf/modules/align_MT.config
@@ -23,12 +23,6 @@ process {
         ext.prefix = { "${meta.id}_sorted" }
     }
 
-    withName: '.*ALIGN_MT:BWAMEME_MEM_MT' {
-        ext.args = { "-7 -M -K 100000000 -R ${meta.read_group}" }
-        ext.args2 = { "-T ./samtools_sort_tmp" }
-        ext.prefix = { "${meta.id}_sorted" }
-    }
-
     withName: '.*ALIGN_MT:BWA_MEM_MT' {
         ext.args = { "-M -K 100000000 -R ${meta.read_group}" }
         ext.args2 = { "-T ./samtools_sort_tmp" }
@@ -80,12 +74,6 @@ process {
         ext.prefix = { "${meta.id}_sorted_shifted" }
     }
 
-    withName: '.*ALIGN_MT_SHIFT:BWAMEME_MEM_MT' {
-        ext.args = { "-7 -M -K 100000000 -R ${meta.read_group}" }
-        ext.args2 = { "-T ./samtools_sort_tmp" }
-        ext.prefix = { "${meta.id}_sorted_shifted" }
-    }
-
     withName: '.*ALIGN_MT_SHIFT:SENTIEON_BWAMEM_MT' {
         ext.args = { "-M -K 10000000 -R ${meta.read_group}" }
         ext.args2 = { "-T ./samtools_sort_tmp" }

diff --git a/conf/modules/annotate_genome_snvs.config b/conf/modules/annotate_genome_snvs.config
@@ -18,6 +18,7 @@
 process {
 
     withName: '.*ANNOTATE_GENOME_SNVS:BCFTOOLS_ROH' {
+        ext.when = { meta.probands.size()>0 }
         ext.args = { "--samples ${meta.probands.unique().first()} --skip-indels " }
         ext.prefix = { "${meta.id}_roh" }
     }
@@ -30,13 +31,18 @@ process {
         ext.prefix = { "${input.simpleName}" }
     }
 
+    withName: '.*ANNOTATE_GENOME_SNVS:GATK4_SELECTVARIANTS' {
+        ext.prefix = { "${vcf.simpleName}_${intervals.simpleName}" }
+    }
+
     withName: '.*ANNOTATE_GENOME_SNVS:VCFANNO' {
-        ext.prefix = { "${vcf.simpleName}_vcfanno" }
+        ext.prefix = { "${meta.prefix}_vcfanno_${meta.scatterid}" }
     }
 
     withName: '.*ANNOTATE_GENOME_SNVS:UPD_SITES' {
         ext.prefix = { "${vcf.simpleName}_upd_sites" }
         ext.args = {"--af-tag GNOMADAF --proband ${meta.upd_child} --mother ${meta.mother} --father ${meta.father} sites"}
+        ext.when = { !(workflow.profile.tokenize(',').intersect(['test', 'test_one_sample']).size() >= 1) || workflow.stubRun }
     }
 
     withName: '.*ANNOTATE_GENOME_SNVS:UPD_REGIONS' {
@@ -73,14 +79,10 @@ process {
     }
 
     withName: '.*ANNOTATE_GENOME_SNVS:BCFTOOLS_VIEW' {
-        ext.prefix = { "${vcf.simpleName}_filter" }
+        ext.prefix = { "${meta.prefix}_filter_${meta.scatterid}" }
         ext.args = { '--output-type z --exclude "INFO/GNOMADAF > 0.70 | INFO/GNOMADAF_popmax > 0.70" ' }
     }
 
-    withName: '.*ANNOTATE_GENOME_SNVS:GATK4_SELECTVARIANTS' {
-        ext.prefix = { "${vcf.simpleName}_${intervals.simpleName}" }
-    }
-
     withName: '.*ANNOTATE_GENOME_SNVS:ENSEMBLVEP_SNV' {
         ext.prefix = { "${meta.prefix}_vep_${meta.scatterid}" }
         ext.args = { [

diff --git a/conf/modules/annotate_repeat_expansions.config b/conf/modules/annotate_repeat_expansions.config
@@ -0,0 +1,41 @@
+/*
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+    Config file for defining DSL2 per module options and publishing paths
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+    Available keys to override module options:
+        ext.args            = Additional arguments appended to command in module.
+        ext.args2           = Second set of arguments appended to command in module (multi-tool modules).
+        ext.args3           = Third set of arguments appended to command in module (multi-tool modules).
+        ext.prefix          = File name prefix for output files.
+        ext.when            = Conditional clause
+----------------------------------------------------------------------------------------
+*/
+
+//
+// Repeat expansion calling options
+//
+
+process {
+
+    withName: '.*ANNOTATE_REPEAT_EXPANSIONS:STRANGER' {
+        ext.prefix = { "${meta.id}_stranger" }
+    }
+
+    withName: '.*ANNOTATE_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
+        ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
+        ext.args   = '--output-type z'
+        publishDir = [
+            path: { "${params.outdir}/repeat_expansions" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
+        ]
+    }
+
+    withName: '.*ANNOTATE_REPEAT_EXPANSIONS:INDEX_STRANGER' {
+        publishDir = [
+            path: { "${params.outdir}/repeat_expansions" },
+            mode: params.publish_dir_mode,
+            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
+        ]
+    }
+}
diff --git a/conf/modules/call_repeat_expansions.config b/conf/modules/call_repeat_expansions.config
@@ -58,25 +58,4 @@ process {
         ext.args = {"--notag"}
     }
 
-    withName: '.*CALL_REPEAT_EXPANSIONS:STRANGER' {
-        ext.prefix = { "${meta.id}_stranger" }
-    }
-
-    withName: '.*CALL_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
-        ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
-        ext.args   = '--output-type z'
-        publishDir = [
-            path: { "${params.outdir}/repeat_expansions" },
-            mode: params.publish_dir_mode,
-            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
-        ]
-    }
-
-    withName: '.*CALL_REPEAT_EXPANSIONS:INDEX_STRANGER' {
-        publishDir = [
-            path: { "${params.outdir}/repeat_expansions" },
-            mode: params.publish_dir_mode,
-            saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
-        ]
-    }
 }
diff --git a/conf/modules/call_snv_deepvariant.config b/conf/modules/call_snv_deepvariant.config
@@ -22,7 +22,10 @@ process {
     }
 
     withName: '.*CALL_SNV_DEEPVARIANT:DEEPVARIANT' {
-        ext.args = { "--model_type=${params.analysis_type.toUpperCase()}" }
+        ext.args = { [
+            "--model_type=${params.analysis_type.toUpperCase()}",
+            meta.sex == "1" ? params.genome == 'GRCh37' ? '--haploid_contigs="X,Y"' : '--haploid_contigs="chrX,chrY"' : ''
+        ].join(' ') }
         ext.prefix = { "${meta.id}_deepvar" }
     }
 

diff --git a/conf/modules/prepare_references.config b/conf/modules/prepare_references.config
@@ -26,37 +26,32 @@ process {
     }
 
     withName: '.*PREPARE_REFERENCES:BWAMEM2_INDEX_GENOME' {
-        ext.when = {!params.bwamem2 && params.aligner == "bwamem2"}
+        ext.when = {!params.bwamem2 && (params.aligner == "bwamem2" || params.mt_aligner == "bwamem2")}
     }
 
     withName: '.*PREPARE_REFERENCES:BWAMEME_INDEX_GENOME' {
         ext.args = '-a meme'
         ext.when = {!params.bwameme && params.aligner == "bwameme"}
     }
 
-    withName: '.*PREPARE_REFERENCES:BWAMEM2_INDEX_MT_SHIFT' {
-        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.aligner == "bwamem2"}
+    withName: '.*PREPARE_REFERENCES:BWA_INDEX_GENOME' {
+        ext.when = {!params.bwa && (!(params.aligner == "sentieon") || params.mt_aligner == "bwa")}
     }
 
-    withName: '.*PREPARE_REFERENCES:BWAMEME_INDEX_MT_SHIFT' {
-        ext.args = '-a meme'
-        ext.when = { !(params.analysis_type == "wes") && params.aligner == "bwameme"}
+    withName: '.*PREPARE_REFERENCES:SENTIEON_BWAINDEX_GENOME' {
+        ext.when = {!params.bwa && (params.aligner == "sentieon" || params.mt_aligner == "sentieon")}
     }
 
-    withName: '.*PREPARE_REFERENCES:SENTIEON_BWAINDEX_GENOME' {
-        ext.when = {!params.bwa && params.aligner == "sentieon"}
+    withName: '.*PREPARE_REFERENCES:BWAMEM2_INDEX_MT_SHIFT' {
+        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.mt_aligner == "bwamem2"}
     }
 
     withName: '.*PREPARE_REFERENCES:SENTIEON_BWAINDEX_MT_SHIFT' {
-        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.aligner == "sentieon"}
+        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.mt_aligner == "sentieon"}
     }
 
     withName: '.*PREPARE_REFERENCES:BWA_INDEX_MT_SHIFT' {
-        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.aligner == "bwa"}
-    }
-
-    withName: '.*PREPARE_REFERENCES:BWA_INDEX_GENOME' {
-        ext.when = {!params.bwa && (!(params.aligner == "sentieon") || params.aligner == "bwa")}
+        ext.when = { (params.analysis_type.equals("wgs") || params.run_mt_for_wes) && params.mt_aligner == "bwa"}
     }
 
     withName: '.*PREPARE_REFERENCES:SAMTOOLS_FAIDX_GENOME' {
@@ -105,6 +100,10 @@ process {
         ext.when = { !params.target_bed.equals(null) && !params.target_bed.endsWith(".gz") }
     }
 
+    withName: '.*PREPARE_REFERENCES:TABIX_BGZIPINDEX_VCFANNOEXTRA' {
+        ext.args2 = '--csi'
+    }
+
     withName: '.*PREPARE_REFERENCES:GATK_BILT' {
         ext.when = { !params.target_bed.equals(null) }
         ext.prefix = { "${meta.id}_target" }
@@ -124,19 +123,4 @@ process {
         ext.when = { (params.vep_cache && params.vep_cache.endsWith("tar.gz")) }
     }
 
-    withName: '.*PREPARE_REFERENCES:GATK_PREPROCESS_WGS' {
-        ext.args =  { [
-            '--padding 0',
-            '--interval-merging-rule OVERLAPPING_ONLY',
-            "--exclude-intervals ${params.mito_name}",
-            "--tmp-dir ./"
-            ].join(' ') }
-        ext.when = { params.analysis_type.equals("wgs") && !params.readcount_intervals }
-    }
-
-    withName: '.*PREPARE_REFERENCES:GATK_PREPROCESS_WES' {
-        ext.args   = { "--bin-length 0 --interval-merging-rule OVERLAPPING_ONLY --exclude-intervals ${params.mito_name}" }
-        ext.when = { params.analysis_type.equals("wes") && !params.readcount_intervals }
-    }
-
 }
diff --git a/conf/modules/qc_bam.config b/conf/modules/qc_bam.config
@@ -35,10 +35,8 @@ process {
         ext.prefix = { "${meta.id}_hsmetrics" }
     }
 
-    if (!params.skip_qualimap) {
-        withName: '.*QC_BAM:QUALIMAP_BAMQC' {
-            ext.prefix = { "${meta.id}_qualimap" }
-        }
+    withName: '.*QC_BAM:QUALIMAP_BAMQC' {
+        ext.prefix = { "${meta.id}_qualimap" }
     }
 
     withName: '.*QC_BAM:TIDDIT_COV' {

diff --git a/conf/modules/raredisease.config b/conf/modules/raredisease.config
@@ -21,6 +21,7 @@ process {
         ext.args = '--quiet --dir ./'
         publishDir = [
             path: { "${params.outdir}/fastqc/${meta.id}" },
+            mode: params.publish_dir_mode,
             saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
         ]
     }
@@ -67,6 +68,7 @@ process {
     withName: '.*RAREDISEASE:CREATE_PEDIGREE_FILE' {
         publishDir = [
             path: { "${params.outdir}/pedigree" },
+            mode: params.publish_dir_mode,
             saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
         ]
     }

diff --git a/conf/modules/scatter_genome.config b/conf/modules/scatter_genome.config
@@ -21,7 +21,7 @@ process {
     }
 
     withName: '.*SCATTER_GENOME:GATK4_SPLITINTERVALS' {
-        ext.args = { "--subdivision-mode BALANCING_WITHOUT_INTERVAL_SUBDIVISION --scatter-count 22" }
+        ext.args = { "--scatter-count ${params.scatter_count}" }
         ext.when = { !params.skip_snv_annotation }
         ext.prefix = { "${meta.id}_genome_intervals" }
         publishDir = [