Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor #574

Merged
merged 7 commits into from
Jun 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,3 @@ jobs:

- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
Comment on lines -72 to -79
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So now neither the stub run nor the regular run of the downloaded pipeline is tested? As a user of an offline cluster, it's very nice to know that nf-core download works smoothly. But if there's no space maybe there's nothing to do.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that's right. The downloaded pipeline is not tested anymore. I can make sure that the downloaded pipeline works locally before a release, but the github runners just don't have enough space for both the downloaded containers and the tests :(

24 changes: 17 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### `Added`

- Two new parameters `skip_repeat_annotation` and `skip_repeat_calling` to skip calling and annotation of repeat expansions [#574](https://github.com/nf-core/raredisease/pull/574)
- A new parameter `skip_smncopynumbercaller` to skip smncopynumbercaller module[#574](https://github.com/nf-core/raredisease/pull/574)
- A new parameter `skip_sv_calling` to skip sv calling workflow [#572](https://github.com/nf-core/raredisease/pull/572)
- Two new parameters `skip_snv_calling` and `skip_repeat_analysis` to skip snv calling and repeat analysis respectively [#571](https://github.com/nf-core/raredisease/pull/571)
- Two new parameters `mbuffer_mem` and `samtools_sort_threads` to control resources given to mbuffer and samtools sort in the bwameme module [#570](https://github.com/nf-core/raredisease/pull/570)

### `Changed`

- Removed `skip_repeat_analysis` added in #571 [#574](https://github.com/nf-core/raredisease/pull/574)
- Remove several skip parameters that had been included in the pipeline to avoid failed CI tests (see parameters table below) [#574](https://github.com/nf-core/raredisease/pull/574)
- `readcount_intervals` parameter is now mandatory for running germlinecnvcaller. [#570](https://github.com/nf-core/raredisease/pull/570)
- Turn off CNVnator, TIDDIT, SMNCopyNumberCaller, Gens, and Vcf2cytosure for targeted analysis [#573](https://github.com/nf-core/raredisease/pull/573)

Expand All @@ -23,13 +27,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Parameters

| Old parameter | New parameter |
| ------------- | --------------------- |
| | mbuffer_mem |
| | samtools_sort_threads |
| | skip_repeat_analysis |
| | skip_snv_calling |
| | skip_sv_calling |
| Old parameter | New parameter |
| --------------- | ------------------------ |
| | mbuffer_mem |
| | samtools_sort_threads |
| | skip_repeat_calling |
| | skip_snv_calling |
| | skip_sv_calling |
| skip_eklipse | |
| skip_fastqc | |
| skip_haplocheck | |
| skip_qualimap | |
| | skip_smncopynumbercaller |
| | skip_repeat_annotation |

## 2.1.0 - Obelix [2024-05-29]

Expand Down
41 changes: 41 additions & 0 deletions conf/modules/annotate_repeat_expansions.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Config file for defining DSL2 per module options and publishing paths
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Available keys to override module options:
ext.args = Additional arguments appended to command in module.
ext.args2 = Second set of arguments appended to command in module (multi-tool modules).
ext.args3 = Third set of arguments appended to command in module (multi-tool modules).
ext.prefix = File name prefix for output files.
ext.when = Conditional clause
----------------------------------------------------------------------------------------
*/

//
// Repeat expansion calling options
//

process {

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:STRANGER' {
ext.prefix = { "${meta.id}_stranger" }
}

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
ext.args = '--output-type z'
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:INDEX_STRANGER' {
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}
}
21 changes: 0 additions & 21 deletions conf/modules/call_repeat_expansions.config
Original file line number Diff line number Diff line change
Expand Up @@ -58,25 +58,4 @@ process {
ext.args = {"--notag"}
}

withName: '.*CALL_REPEAT_EXPANSIONS:STRANGER' {
ext.prefix = { "${meta.id}_stranger" }
}

withName: '.*CALL_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
ext.args = '--output-type z'
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}

withName: '.*CALL_REPEAT_EXPANSIONS:INDEX_STRANGER' {
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}
}
6 changes: 2 additions & 4 deletions conf/modules/qc_bam.config
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,8 @@ process {
ext.prefix = { "${meta.id}_hsmetrics" }
}

if (!params.skip_qualimap) {
withName: '.*QC_BAM:QUALIMAP_BAMQC' {
ext.prefix = { "${meta.id}_qualimap" }
}
withName: '.*QC_BAM:QUALIMAP_BAMQC' {
ext.prefix = { "${meta.id}_qualimap" }
}

withName: '.*QC_BAM:TIDDIT_COV' {
Expand Down
4 changes: 0 additions & 4 deletions conf/test.config
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,7 @@ params {
mito_name = 'MT'

// analysis params
skip_eklipse = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_fastqc = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_germlinecnvcaller = true
skip_haplocheck = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_qualimap = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_mt_annotation = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip annotation on Github CI
skip_mt_subsample = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip subsample on Github CI
skip_peddy = true
Expand Down
4 changes: 0 additions & 4 deletions conf/test_one_sample.config
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,7 @@ params {
mito_name = 'MT'

// analysis params
skip_eklipse = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_fastqc = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_germlinecnvcaller = true
skip_haplocheck = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_qualimap = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_mt_annotation = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip annotation on Github CI
skip_mt_subsample = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip subsample on Github CI
skip_peddy = true
Expand Down
9 changes: 4 additions & 5 deletions nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,16 @@ params {
run_mt_for_wes = false
run_rtgvcfeval = false
save_mapped_as_cram = false
skip_eklipse = false
skip_fastp = false
skip_fastqc = false
skip_gens = true
skip_germlinecnvcaller = false
skip_haplocheck = false
skip_peddy = false
skip_me_calling = false
skip_me_annotation = false
skip_mt_annotation = false
skip_qualimap = false
skip_repeat_analysis = false
skip_repeat_annotation = false
skip_repeat_calling = false
skip_smncopynumbercaller = false
skip_snv_annotation = false
skip_snv_calling = false
skip_sv_annotation = false
Expand Down Expand Up @@ -319,6 +317,7 @@ includeConfig 'conf/modules/annotate_consequence_pli.config'
includeConfig 'conf/modules/annotate_genome_snvs.config'
includeConfig 'conf/modules/annotate_mt_snvs.config'
includeConfig 'conf/modules/annotate_structural_variants.config'
includeConfig 'conf/modules/annotate_repeat_expansions.config'
includeConfig 'conf/modules/call_repeat_expansions.config'
includeConfig 'conf/modules/call_snv.config'
includeConfig 'conf/modules/call_structural_variants.config'
Expand Down
34 changes: 12 additions & 22 deletions nextflow_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -508,21 +508,11 @@
"description": "Specifies whether to generate and publish alignment files as cram instead of bam",
"fa_icon": "fas fa-toggle-on"
},
"skip_fastqc": {
"type": "boolean",
"description": "Specifies whether or not to skip FASTQC.",
"fa_icon": "fas fa-toggle-on"
},
"skip_fastp": {
"type": "boolean",
"description": "Specifies whether or not to skip trimming with fastp.",
"fa_icon": "fas fa-toggle-on"
},
"skip_haplocheck": {
"type": "boolean",
"description": "Specifies whether or not to skip haplocheck.",
"fa_icon": "fas fa-toggle-on"
},
"skip_gens": {
"type": "boolean",
"description": "Specifies whether or not to skip gens preprocessing subworkflow.",
Expand All @@ -533,21 +523,11 @@
"description": "Specifies whether or not to skip CNV calling using GATK's GermlineCNVCaller",
"fa_icon": "fas fa-toggle-on"
},
"skip_eklipse": {
"type": "boolean",
"description": "Specifies whether or not to skip eKLIPse.",
"fa_icon": "fas fa-toggle-on"
},
"skip_peddy": {
"type": "boolean",
"description": "Specifies whether or not to skip peddy.",
"fa_icon": "fas fa-toggle-on"
},
"skip_qualimap": {
"type": "boolean",
"description": "Specifies whether or not to skip Qualimap.",
"fa_icon": "fas fa-toggle-on"
},
"skip_me_calling": {
"type": "boolean",
"description": "Specifies whether or not to skip calling mobile elements, and the subsequent annotation step.",
Expand All @@ -568,9 +548,19 @@
"description": "Specifies whether or not to subsample mt alignment.",
"fa_icon": "fas fa-toggle-on"
},
"skip_repeat_analysis": {
"skip_repeat_annotation": {
"type": "boolean",
"description": "Specifies whether or not to skip annotation of repeat expansions.",
"fa_icon": "fas fa-toggle-on"
},
"skip_repeat_calling": {
"type": "boolean",
"description": "Specifies whether or not to skip calling of repeat expansions.",
"fa_icon": "fas fa-toggle-on"
},
"skip_smncopynumbercaller": {
"type": "boolean",
"description": "Specifies whether or not to skip calling and annotation of repeat expansions.",
"description": "Specifies whether or not to skip smncopynumbercaller.",
"fa_icon": "fas fa-toggle-on"
},
"skip_snv_annotation": {
Expand Down
34 changes: 34 additions & 0 deletions subworkflows/local/annotate_repeat_expansions.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
//
// Annotate repeat expansions
//

include { BCFTOOLS_VIEW as COMPRESS_STRANGER } from '../../modules/nf-core/bcftools/view/main'
include { STRANGER } from '../../modules/nf-core/stranger/main'
include { TABIX_TABIX as INDEX_STRANGER } from '../../modules/nf-core/tabix/tabix/main'

workflow ANNOTATE_REPEAT_EXPANSIONS {
take:
ch_variant_catalog // channel: [mandatory] [ path(variant_catalog.json) ]
ch_vcf // channel: [mandatory] [ val(meta), path(vcf) ]

main:
ch_versions = Channel.empty()

// Annotate, compress and index
STRANGER ( ch_vcf, ch_variant_catalog )
COMPRESS_STRANGER (
STRANGER.out.vcf.map{ meta, vcf -> [meta, vcf, [] ]},
[], [], []
)
INDEX_STRANGER ( COMPRESS_STRANGER.out.vcf )

ch_vcf_idx = COMPRESS_STRANGER.out.vcf.join(INDEX_STRANGER.out.tbi, failOnMismatch:true, failOnDuplicate:true)

ch_versions = ch_versions.mix(STRANGER.out.versions.first())
ch_versions = ch_versions.mix(COMPRESS_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(INDEX_STRANGER.out.versions.first())

emit:
vcf = ch_vcf_idx // channel: [ val(meta), path(vcf), path(tbi) ]
versions = ch_versions // channel: [ path(versions.yml) ]
}
20 changes: 2 additions & 18 deletions subworkflows/local/call_repeat_expansions.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,11 @@

include { BCFTOOLS_NORM as SPLIT_MULTIALLELICS_EXP } from '../../modules/nf-core/bcftools/norm/main'
include { BCFTOOLS_REHEADER as BCFTOOLS_REHEADER_EXP } from '../../modules/nf-core/bcftools/reheader/main'
include { BCFTOOLS_VIEW as COMPRESS_STRANGER } from '../../modules/nf-core/bcftools/view/main'
include { EXPANSIONHUNTER } from '../../modules/nf-core/expansionhunter/main'
include { PICARD_RENAMESAMPLEINVCF as RENAMESAMPLE_EXP } from '../../modules/nf-core/picard/renamesampleinvcf/main'
include { STRANGER } from '../../modules/nf-core/stranger/main'
include { SAMTOOLS_SORT } from '../../modules/nf-core/samtools/sort/main'
include { SAMTOOLS_INDEX } from '../../modules/nf-core/samtools/index/main'
include { SVDB_MERGE as SVDB_MERGE_REPEATS } from '../../modules/nf-core/svdb/merge/main'
include { TABIX_TABIX as INDEX_STRANGER } from '../../modules/nf-core/tabix/tabix/main'
include { TABIX_TABIX as TABIX_EXP_RENAME } from '../../modules/nf-core/tabix/tabix/main'

workflow CALL_REPEAT_EXPANSIONS {
Expand Down Expand Up @@ -63,29 +60,16 @@ workflow CALL_REPEAT_EXPANSIONS {

SVDB_MERGE_REPEATS ( ch_svdb_merge_input, [] )

// Annotate, compress and index
STRANGER ( SVDB_MERGE_REPEATS.out.vcf, ch_variant_catalog )
COMPRESS_STRANGER (
STRANGER.out.vcf.map{ meta, vcf -> [meta, vcf, [] ]},
[], [], []
)
INDEX_STRANGER ( COMPRESS_STRANGER.out.vcf )

ch_vcf_idx = COMPRESS_STRANGER.out.vcf.join(INDEX_STRANGER.out.tbi, failOnMismatch:true, failOnDuplicate:true)

ch_versions = ch_versions.mix(EXPANSIONHUNTER.out.versions.first())
ch_versions = ch_versions.mix(BCFTOOLS_REHEADER_EXP.out.versions.first())
ch_versions = ch_versions.mix(RENAMESAMPLE_EXP.out.versions.first() )
ch_versions = ch_versions.mix(TABIX_EXP_RENAME.out.versions.first())
ch_versions = ch_versions.mix(SPLIT_MULTIALLELICS_EXP.out.versions.first())
ch_versions = ch_versions.mix(SVDB_MERGE_REPEATS.out.versions.first())
ch_versions = ch_versions.mix(STRANGER.out.versions.first())
ch_versions = ch_versions.mix(COMPRESS_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(INDEX_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(SAMTOOLS_SORT.out.versions.first())
ch_versions = ch_versions.mix(SAMTOOLS_INDEX.out.versions.first())

emit:
vcf = ch_vcf_idx // channel: [ val(meta), path(vcf), path(tbi) ]
versions = ch_versions // channel: [ path(versions.yml) ]
vcf = SVDB_MERGE_REPEATS.out.vcf // channel: [ val(meta), path(vcf) ]
versions = ch_versions // channel: [ path(versions.yml) ]
}
6 changes: 2 additions & 4 deletions subworkflows/local/qc_bam.nf
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,8 @@ workflow QC_BAM {

PICARD_COLLECTHSMETRICS (ch_hsmetrics_in, ch_genome_fasta, ch_genome_fai, [[],[]])

if (!params.skip_qualimap) {
ch_qualimap = QUALIMAP_BAMQC (ch_bam, []).results
ch_versions = ch_versions.mix(QUALIMAP_BAMQC.out.versions.first())
}
ch_qualimap = QUALIMAP_BAMQC (ch_bam, []).results
ch_versions = ch_versions.mix(QUALIMAP_BAMQC.out.versions.first())

TIDDIT_COV (ch_bam, [[],[]]) // 2nd pos. arg is req. only for cram input

Expand Down
Loading