More complete CI #806

maxulysse · 2022-11-02T13:17:10Z

Following @emiller88 on nf-core/rnaseq#883

PR checklist

This comment contains a description of changes (with reason).
If you've fixed a bug or added code that should be tested, add tests!
If you've added a new tool - have you followed the pipeline conventions in the contribution docs- [ ] If necessary, also make a PR on the nf-core/sarek branch on the nf-core/test-datasets repository.
Make sure your code lints (nf-core lint).
Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
Usage Documentation in docs/usage.md is updated.
Output Documentation in docs/output.md is updated.
CHANGELOG.md is updated.
README.md is updated (including new tool citations and authors/contributors).

github-actions · 2022-11-02T13:18:45Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 93f4db2

+| ✅ 152 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗   1 tests had warnings |!

❗ Test warnings:

pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your prefered methods description, e.g. add publication citation for this pipeline

❔ Tests ignored:

files_exist - File is ignored: conf/modules.config
files_exist - File is ignored: conf/test.config
files_exist - File is ignored: conf/test_full.config
files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_light.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo_dark.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_light.png
files_exist - File found: docs/images/nf-core-sarek_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-sarek_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.version
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '3.1dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 21.10.3, Config: 21.10.3
readme - README Nextflow minimum version in Quick Start section matched config. README: 21.10.3, Config: 21.10.3
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (443 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: pytest-workflow.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: awsfulltest_germline.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: awstest.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.6
Run at 2022-11-04 16:44:17

maxulysse · 2022-11-03T16:11:40Z

Launching all tests with singularity and conda and will add exclude in the GHA workflow to remove the ones that don't work (thinking mainly about conda + annotation)

edmundmiller

Awesome work and a massive effort!

Only thought is you might consider splitting it into a workflows directory or something to split it apart from subworkflows and modules for other local tests in the future, but that can be addressed when we get there.

Also, you might consider not running singularity and conda on every commit to a PR as well, there are a lot of tests here. So not only taking up a ton of runners and energy, but you might hit the matrix limit.

.github/workflows/pytest-workflow.yml

edmundmiller · 2022-11-03T23:11:15Z

tests/test_cnvkit.yml

@@ -0,0 +1,158 @@
+- name: Run variant calling on somatic samples with cnvkit


Might consider splitting these into their own individual files tests/workflows/cnvkit/test_variantcalling_somatic.yml or something for readability, these are going to get quite long.

I don't think there will be much more tests for cnvkit actually, but I'll think about that.
Thanks

maxulysse · 2022-11-04T09:55:53Z

Awesome work and a massive effort!

Only thought is you might consider splitting it into a workflows directory or something to split it apart from subworkflows and modules for other local tests in the future, but that can be addressed when we get there.

Also, you might consider not running singularity and conda on every commit to a PR as well, there are a lot of tests here. So not only taking up a ton of runners and energy, but you might hit the matrix limit.

Was thinking more not to run the modules or subworkflows tests as I'm assuming we'll run that in nf-core/modules.
Maybe just for the local ones, but will need to figure this out....

edmundmiller · 2022-11-04T14:38:22Z

I think those will come with nf-test. It's a lot speedier to test with than pytest, so testing your local modules and subworkflows is a lot easier.

edmundmiller · 2022-11-04T14:42:05Z

BTW, sorry for figuring this out just as you set this up 🙈 https://github.com/nf-core/methylseq/blob/dev/.github/workflows/ci.yml

FriederikeHanssen

Love the splitting of the modules.config

starting to reorganize conf files

e06695f

maxulysse added 28 commits November 2, 2022 14:20

fix lint

5fee072

fix lint

c2f6ee9

add first tests

2e728fc

fix path

a0173b8

fix workflow

72b4140

fixing aligner tests

34965c0

fix aligner tests

1085042

update annotation tests

3122950

test only with docker for now

68da3de

testing out adding an extra layer of tags

4c05d1c

add info as comments instead

986fe91

sort files

b9de3cf

fix file names

7715760

alignement_to_fastq

630eab9

markduplicates

767c9ef

rename files

66d770b

prepare_recalibration + recalibrate

83ed74a

Merge remote-tracking branch 'upstream/dev' into dev_more_complete_CI

13ab016

code polish + intervals tests

9fc7b31

prettier . -w

a3f502c

recalibrate

c5a4531

trimming

b6ea03e

umi

e14cc7f

default + trimming | split fastq > fastp

4a3685d

target > intervals

bfc72f6

pair > default

0cc8fac

cnvkit

20ba4bd

controlfreec

10c74f1

maxulysse added 6 commits November 3, 2022 15:20

deepvariant

2298738

freebayes

696796b

haplotypecaller

e136625

manta

a922a39

mpileup + controlfreec

7f56a74

mutect2

ec13c41

edmundmiller self-requested a review November 3, 2022 15:37

msisensorpro + strelka + strelka_bp + tiddit

cfc634f

maxulysse marked this pull request as ready for review November 3, 2022 15:43

test with singularity and conda as well

8ca37fa

maxulysse added 8 commits November 3, 2022 17:30

update GHA workflows

1694983

remove tags

09241c5

remove more conda tests

7faac67

remove more tags

5c6b995

update default

e8c20a1

less default tests

9184570

less default tests

819d284

exclude conda from annotation tests

f495369

edmundmiller approved these changes Nov 3, 2022

View reviewed changes

exclude more tests + remove md5sum for some conda tests

c31691f

maxulysse added 2 commits November 4, 2022 12:20

more conda md5sum removed

973bbcb

comments

8af8cc4

more conda md5sum removed + remove duplicate tests

93f4db2

FriederikeHanssen approved these changes Nov 4, 2022

View reviewed changes

maxulysse merged commit 40050f7 into nf-core:dev Nov 4, 2022

maxulysse deleted the dev_more_complete_CI branch November 4, 2022 18:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

More complete CI #806

More complete CI #806

maxulysse commented Nov 2, 2022

github-actions bot commented Nov 2, 2022 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse commented Nov 3, 2022

edmundmiller left a comment •

edited

Loading

edmundmiller Nov 3, 2022

maxulysse Nov 4, 2022

maxulysse commented Nov 4, 2022

edmundmiller commented Nov 4, 2022

edmundmiller commented Nov 4, 2022

FriederikeHanssen left a comment

		@@ -0,0 +1,158 @@
		- name: Run variant calling on somatic samples with cnvkit

More complete CI #806

More complete CI #806

Conversation

maxulysse commented Nov 2, 2022

PR checklist

github-actions bot commented Nov 2, 2022 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

maxulysse commented Nov 3, 2022

edmundmiller left a comment • edited Loading

Choose a reason for hiding this comment

edmundmiller Nov 3, 2022

Choose a reason for hiding this comment

maxulysse Nov 4, 2022

Choose a reason for hiding this comment

maxulysse commented Nov 4, 2022

edmundmiller commented Nov 4, 2022

edmundmiller commented Nov 4, 2022

FriederikeHanssen left a comment

Choose a reason for hiding this comment

github-actions bot commented Nov 2, 2022 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️

edmundmiller left a comment •

edited

Loading