Skip to content

Commit

Permalink
Merge pull request #574 from nf-core/change-download-ci
Browse files Browse the repository at this point in the history
Refactor
  • Loading branch information
ramprasadn authored Jun 26, 2024
2 parents 27fde73 + 8101033 commit 427cbef
Show file tree
Hide file tree
Showing 16 changed files with 151 additions and 126 deletions.
8 changes: 0 additions & 8 deletions .github/workflows/download_pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,3 @@ jobs:
- name: Inspect download
run: tree ./${{ env.REPOTITLE_LOWERCASE }}

- name: Run the downloaded pipeline (stub)
id: stub_run_pipeline
continue-on-error: true
env:
NXF_SINGULARITY_CACHEDIR: ./
NXF_SINGULARITY_HOME_MOUNT: true
run: nextflow run ./${{ env.REPOTITLE_LOWERCASE }}/$( sed 's/\W/_/g' <<< ${{ env.REPO_BRANCH }}) -stub -profile test,singularity --outdir ./results
24 changes: 17 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### `Added`

- Two new parameters `skip_repeat_annotation` and `skip_repeat_calling` to skip calling and annotation of repeat expansions [#574](https://github.com/nf-core/raredisease/pull/574)
- A new parameter `skip_smncopynumbercaller` to skip smncopynumbercaller module[#574](https://github.com/nf-core/raredisease/pull/574)
- A new parameter `skip_sv_calling` to skip sv calling workflow [#572](https://github.com/nf-core/raredisease/pull/572)
- Two new parameters `skip_snv_calling` and `skip_repeat_analysis` to skip snv calling and repeat analysis respectively [#571](https://github.com/nf-core/raredisease/pull/571)
- Two new parameters `mbuffer_mem` and `samtools_sort_threads` to control resources given to mbuffer and samtools sort in the bwameme module [#570](https://github.com/nf-core/raredisease/pull/570)

### `Changed`

- Removed `skip_repeat_analysis` added in #571 [#574](https://github.com/nf-core/raredisease/pull/574)
- Remove several skip parameters that had been included in the pipeline to avoid failed CI tests (see parameters table below) [#574](https://github.com/nf-core/raredisease/pull/574)
- `readcount_intervals` parameter is now mandatory for running germlinecnvcaller. [#570](https://github.com/nf-core/raredisease/pull/570)
- Turn off CNVnator, TIDDIT, SMNCopyNumberCaller, Gens, and Vcf2cytosure for targeted analysis [#573](https://github.com/nf-core/raredisease/pull/573)

Expand All @@ -23,13 +27,19 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Parameters

| Old parameter | New parameter |
| ------------- | --------------------- |
| | mbuffer_mem |
| | samtools_sort_threads |
| | skip_repeat_analysis |
| | skip_snv_calling |
| | skip_sv_calling |
| Old parameter | New parameter |
| --------------- | ------------------------ |
| | mbuffer_mem |
| | samtools_sort_threads |
| | skip_repeat_calling |
| | skip_snv_calling |
| | skip_sv_calling |
| skip_eklipse | |
| skip_fastqc | |
| skip_haplocheck | |
| skip_qualimap | |
| | skip_smncopynumbercaller |
| | skip_repeat_annotation |

## 2.1.0 - Obelix [2024-05-29]

Expand Down
41 changes: 41 additions & 0 deletions conf/modules/annotate_repeat_expansions.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Config file for defining DSL2 per module options and publishing paths
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Available keys to override module options:
ext.args = Additional arguments appended to command in module.
ext.args2 = Second set of arguments appended to command in module (multi-tool modules).
ext.args3 = Third set of arguments appended to command in module (multi-tool modules).
ext.prefix = File name prefix for output files.
ext.when = Conditional clause
----------------------------------------------------------------------------------------
*/

//
// Repeat expansion calling options
//

process {

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:STRANGER' {
ext.prefix = { "${meta.id}_stranger" }
}

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
ext.args = '--output-type z'
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}

withName: '.*ANNOTATE_REPEAT_EXPANSIONS:INDEX_STRANGER' {
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}
}
21 changes: 0 additions & 21 deletions conf/modules/call_repeat_expansions.config
Original file line number Diff line number Diff line change
Expand Up @@ -58,25 +58,4 @@ process {
ext.args = {"--notag"}
}

withName: '.*CALL_REPEAT_EXPANSIONS:STRANGER' {
ext.prefix = { "${meta.id}_stranger" }
}

withName: '.*CALL_REPEAT_EXPANSIONS:COMPRESS_STRANGER' {
ext.prefix = { "${meta.id}_repeat_expansion_stranger" }
ext.args = '--output-type z'
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}

withName: '.*CALL_REPEAT_EXPANSIONS:INDEX_STRANGER' {
publishDir = [
path: { "${params.outdir}/repeat_expansions" },
mode: params.publish_dir_mode,
saveAs: { filename -> filename.equals('versions.yml') ? null : filename },
]
}
}
6 changes: 2 additions & 4 deletions conf/modules/qc_bam.config
Original file line number Diff line number Diff line change
Expand Up @@ -35,10 +35,8 @@ process {
ext.prefix = { "${meta.id}_hsmetrics" }
}

if (!params.skip_qualimap) {
withName: '.*QC_BAM:QUALIMAP_BAMQC' {
ext.prefix = { "${meta.id}_qualimap" }
}
withName: '.*QC_BAM:QUALIMAP_BAMQC' {
ext.prefix = { "${meta.id}_qualimap" }
}

withName: '.*QC_BAM:TIDDIT_COV' {
Expand Down
4 changes: 0 additions & 4 deletions conf/test.config
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,7 @@ params {
mito_name = 'MT'

// analysis params
skip_eklipse = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_fastqc = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_germlinecnvcaller = true
skip_haplocheck = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_qualimap = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_mt_annotation = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip annotation on Github CI
skip_mt_subsample = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip subsample on Github CI
skip_peddy = true
Expand Down
4 changes: 0 additions & 4 deletions conf/test_one_sample.config
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,7 @@ params {
mito_name = 'MT'

// analysis params
skip_eklipse = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_fastqc = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_germlinecnvcaller = true
skip_haplocheck = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_qualimap = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip tool on Github CI
skip_mt_annotation = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip annotation on Github CI
skip_mt_subsample = System.getenv("GITHUB_ACTIONS").equals(null) ? false : true // skip subsample on Github CI
skip_peddy = true
Expand Down
9 changes: 4 additions & 5 deletions nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -27,18 +27,16 @@ params {
run_mt_for_wes = false
run_rtgvcfeval = false
save_mapped_as_cram = false
skip_eklipse = false
skip_fastp = false
skip_fastqc = false
skip_gens = true
skip_germlinecnvcaller = false
skip_haplocheck = false
skip_peddy = false
skip_me_calling = false
skip_me_annotation = false
skip_mt_annotation = false
skip_qualimap = false
skip_repeat_analysis = false
skip_repeat_annotation = false
skip_repeat_calling = false
skip_smncopynumbercaller = false
skip_snv_annotation = false
skip_snv_calling = false
skip_sv_annotation = false
Expand Down Expand Up @@ -319,6 +317,7 @@ includeConfig 'conf/modules/annotate_consequence_pli.config'
includeConfig 'conf/modules/annotate_genome_snvs.config'
includeConfig 'conf/modules/annotate_mt_snvs.config'
includeConfig 'conf/modules/annotate_structural_variants.config'
includeConfig 'conf/modules/annotate_repeat_expansions.config'
includeConfig 'conf/modules/call_repeat_expansions.config'
includeConfig 'conf/modules/call_snv.config'
includeConfig 'conf/modules/call_structural_variants.config'
Expand Down
34 changes: 12 additions & 22 deletions nextflow_schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -508,21 +508,11 @@
"description": "Specifies whether to generate and publish alignment files as cram instead of bam",
"fa_icon": "fas fa-toggle-on"
},
"skip_fastqc": {
"type": "boolean",
"description": "Specifies whether or not to skip FASTQC.",
"fa_icon": "fas fa-toggle-on"
},
"skip_fastp": {
"type": "boolean",
"description": "Specifies whether or not to skip trimming with fastp.",
"fa_icon": "fas fa-toggle-on"
},
"skip_haplocheck": {
"type": "boolean",
"description": "Specifies whether or not to skip haplocheck.",
"fa_icon": "fas fa-toggle-on"
},
"skip_gens": {
"type": "boolean",
"description": "Specifies whether or not to skip gens preprocessing subworkflow.",
Expand All @@ -533,21 +523,11 @@
"description": "Specifies whether or not to skip CNV calling using GATK's GermlineCNVCaller",
"fa_icon": "fas fa-toggle-on"
},
"skip_eklipse": {
"type": "boolean",
"description": "Specifies whether or not to skip eKLIPse.",
"fa_icon": "fas fa-toggle-on"
},
"skip_peddy": {
"type": "boolean",
"description": "Specifies whether or not to skip peddy.",
"fa_icon": "fas fa-toggle-on"
},
"skip_qualimap": {
"type": "boolean",
"description": "Specifies whether or not to skip Qualimap.",
"fa_icon": "fas fa-toggle-on"
},
"skip_me_calling": {
"type": "boolean",
"description": "Specifies whether or not to skip calling mobile elements, and the subsequent annotation step.",
Expand All @@ -568,9 +548,19 @@
"description": "Specifies whether or not to subsample mt alignment.",
"fa_icon": "fas fa-toggle-on"
},
"skip_repeat_analysis": {
"skip_repeat_annotation": {
"type": "boolean",
"description": "Specifies whether or not to skip annotation of repeat expansions.",
"fa_icon": "fas fa-toggle-on"
},
"skip_repeat_calling": {
"type": "boolean",
"description": "Specifies whether or not to skip calling of repeat expansions.",
"fa_icon": "fas fa-toggle-on"
},
"skip_smncopynumbercaller": {
"type": "boolean",
"description": "Specifies whether or not to skip calling and annotation of repeat expansions.",
"description": "Specifies whether or not to skip smncopynumbercaller.",
"fa_icon": "fas fa-toggle-on"
},
"skip_snv_annotation": {
Expand Down
34 changes: 34 additions & 0 deletions subworkflows/local/annotate_repeat_expansions.nf
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
//
// Annotate repeat expansions
//

include { BCFTOOLS_VIEW as COMPRESS_STRANGER } from '../../modules/nf-core/bcftools/view/main'
include { STRANGER } from '../../modules/nf-core/stranger/main'
include { TABIX_TABIX as INDEX_STRANGER } from '../../modules/nf-core/tabix/tabix/main'

workflow ANNOTATE_REPEAT_EXPANSIONS {
take:
ch_variant_catalog // channel: [mandatory] [ path(variant_catalog.json) ]
ch_vcf // channel: [mandatory] [ val(meta), path(vcf) ]

main:
ch_versions = Channel.empty()

// Annotate, compress and index
STRANGER ( ch_vcf, ch_variant_catalog )
COMPRESS_STRANGER (
STRANGER.out.vcf.map{ meta, vcf -> [meta, vcf, [] ]},
[], [], []
)
INDEX_STRANGER ( COMPRESS_STRANGER.out.vcf )

ch_vcf_idx = COMPRESS_STRANGER.out.vcf.join(INDEX_STRANGER.out.tbi, failOnMismatch:true, failOnDuplicate:true)

ch_versions = ch_versions.mix(STRANGER.out.versions.first())
ch_versions = ch_versions.mix(COMPRESS_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(INDEX_STRANGER.out.versions.first())

emit:
vcf = ch_vcf_idx // channel: [ val(meta), path(vcf), path(tbi) ]
versions = ch_versions // channel: [ path(versions.yml) ]
}
20 changes: 2 additions & 18 deletions subworkflows/local/call_repeat_expansions.nf
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,11 @@

include { BCFTOOLS_NORM as SPLIT_MULTIALLELICS_EXP } from '../../modules/nf-core/bcftools/norm/main'
include { BCFTOOLS_REHEADER as BCFTOOLS_REHEADER_EXP } from '../../modules/nf-core/bcftools/reheader/main'
include { BCFTOOLS_VIEW as COMPRESS_STRANGER } from '../../modules/nf-core/bcftools/view/main'
include { EXPANSIONHUNTER } from '../../modules/nf-core/expansionhunter/main'
include { PICARD_RENAMESAMPLEINVCF as RENAMESAMPLE_EXP } from '../../modules/nf-core/picard/renamesampleinvcf/main'
include { STRANGER } from '../../modules/nf-core/stranger/main'
include { SAMTOOLS_SORT } from '../../modules/nf-core/samtools/sort/main'
include { SAMTOOLS_INDEX } from '../../modules/nf-core/samtools/index/main'
include { SVDB_MERGE as SVDB_MERGE_REPEATS } from '../../modules/nf-core/svdb/merge/main'
include { TABIX_TABIX as INDEX_STRANGER } from '../../modules/nf-core/tabix/tabix/main'
include { TABIX_TABIX as TABIX_EXP_RENAME } from '../../modules/nf-core/tabix/tabix/main'

workflow CALL_REPEAT_EXPANSIONS {
Expand Down Expand Up @@ -63,29 +60,16 @@ workflow CALL_REPEAT_EXPANSIONS {

SVDB_MERGE_REPEATS ( ch_svdb_merge_input, [] )

// Annotate, compress and index
STRANGER ( SVDB_MERGE_REPEATS.out.vcf, ch_variant_catalog )
COMPRESS_STRANGER (
STRANGER.out.vcf.map{ meta, vcf -> [meta, vcf, [] ]},
[], [], []
)
INDEX_STRANGER ( COMPRESS_STRANGER.out.vcf )

ch_vcf_idx = COMPRESS_STRANGER.out.vcf.join(INDEX_STRANGER.out.tbi, failOnMismatch:true, failOnDuplicate:true)

ch_versions = ch_versions.mix(EXPANSIONHUNTER.out.versions.first())
ch_versions = ch_versions.mix(BCFTOOLS_REHEADER_EXP.out.versions.first())
ch_versions = ch_versions.mix(RENAMESAMPLE_EXP.out.versions.first() )
ch_versions = ch_versions.mix(TABIX_EXP_RENAME.out.versions.first())
ch_versions = ch_versions.mix(SPLIT_MULTIALLELICS_EXP.out.versions.first())
ch_versions = ch_versions.mix(SVDB_MERGE_REPEATS.out.versions.first())
ch_versions = ch_versions.mix(STRANGER.out.versions.first())
ch_versions = ch_versions.mix(COMPRESS_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(INDEX_STRANGER.out.versions.first())
ch_versions = ch_versions.mix(SAMTOOLS_SORT.out.versions.first())
ch_versions = ch_versions.mix(SAMTOOLS_INDEX.out.versions.first())

emit:
vcf = ch_vcf_idx // channel: [ val(meta), path(vcf), path(tbi) ]
versions = ch_versions // channel: [ path(versions.yml) ]
vcf = SVDB_MERGE_REPEATS.out.vcf // channel: [ val(meta), path(vcf) ]
versions = ch_versions // channel: [ path(versions.yml) ]
}
6 changes: 2 additions & 4 deletions subworkflows/local/qc_bam.nf
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,8 @@ workflow QC_BAM {

PICARD_COLLECTHSMETRICS (ch_hsmetrics_in, ch_genome_fasta, ch_genome_fai, [[],[]])

if (!params.skip_qualimap) {
ch_qualimap = QUALIMAP_BAMQC (ch_bam, []).results
ch_versions = ch_versions.mix(QUALIMAP_BAMQC.out.versions.first())
}
ch_qualimap = QUALIMAP_BAMQC (ch_bam, []).results
ch_versions = ch_versions.mix(QUALIMAP_BAMQC.out.versions.first())

TIDDIT_COV (ch_bam, [[],[]]) // 2nd pos. arg is req. only for cram input

Expand Down
Loading

0 comments on commit 427cbef

Please sign in to comment.