Add small files to tower.yml

[LilyAnderssonLee]

Add output files information to the file tower.yml.

Close #373

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the pipeline conventions in the contribution docs
• If necessary, also make a PR on the nf-core/taxprofiler branch on the nf-core/test-datasets repository.
• Make sure your code lints (nf-core lint).
• Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
• Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
• Usage Documentation in docs/usage.md is updated.
• Output Documentation in docs/output.md is updated.
• CHANGELOG.md is updated.
• README.md is updated (including new tool citations and authors/contributors).

[github-actions]

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit fce5e6f

+| ✅ 261 tests passed       |+
!| ❗   2 tests had warnings |!

❗ Test warnings:

• pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
• pipeline_todos - TODO string in ci.yml: You can customise CI pipeline run tests as required

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: CODE_OF_CONDUCT.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/.dockstore.yml
• files_exist - File found: .github/CONTRIBUTING.md
• files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: assets/email_template.html
• files_exist - File found: assets/email_template.txt
• files_exist - File found: assets/sendmail_template.txt
• files_exist - File found: assets/nf-core-taxprofiler_logo_light.png
• files_exist - File found: conf/modules.config
• files_exist - File found: conf/test.config
• files_exist - File found: conf/test_full.config
• files_exist - File found: docs/images/nf-core-taxprofiler_logo_light.png
• files_exist - File found: docs/images/nf-core-taxprofiler_logo_dark.png
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: main.nf
• files_exist - File found: assets/multiqc_config.yml
• files_exist - File found: conf/base.config
• files_exist - File found: conf/igenomes.config
• files_exist - File found: .github/workflows/awstest.yml
• files_exist - File found: .github/workflows/awsfulltest.yml
• files_exist - File found: modules.json
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: docs/images/nf-core-taxprofiler_logo.png
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/NfcoreTemplate.groovy
• files_exist - File not found check: lib/Utils.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/WorkflowMain.groovy
• files_exist - File not found check: lib/WorkflowTaxprofiler.groovy
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: Singularity
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Config variable found: manifest.name
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: manifest.homePage
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: process.cpus
• nextflow_config - Config variable found: process.memory
• nextflow_config - Config variable found: process.time
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: params.validationShowHiddenParams
• nextflow_config - Config variable found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config manifest.name began with nf-core/
• nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - Config manifest.version ends in dev: 1.1.8dev
• nextflow_config - Config params.custom_config_version is set to master
• nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Lines for loading custom profiles found
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.preprocessing_qc_tool= fastqc
• nextflow_config - Config default value correct: params.shortread_qc_tool= fastp
• nextflow_config - Config default value correct: params.shortread_qc_minlength= 15
• nextflow_config - Config default value correct: params.shortread_complexityfilter_tool= bbduk
• nextflow_config - Config default value correct: params.shortread_complexityfilter_entropy= 0.3
• nextflow_config - Config default value correct: params.shortread_complexityfilter_bbduk_windowsize= 50
• nextflow_config - Config default value correct: params.shortread_complexityfilter_fastp_threshold= 30
• nextflow_config - Config default value correct: params.shortread_complexityfilter_prinseqplusplus_mode= entropy
• nextflow_config - Config default value correct: params.shortread_complexityfilter_prinseqplusplus_dustscore= 0.5
• nextflow_config - Config default value correct: params.longread_qc_qualityfilter_minlength= 1000
• nextflow_config - Config default value correct: params.longread_qc_qualityfilter_keeppercent= 90
• nextflow_config - Config default value correct: params.longread_qc_qualityfilter_targetbases= 500000000
• nextflow_config - Config default value correct: params.shortread_redundancyestimation_mode= kmer
• nextflow_config - Config default value correct: params.diamond_output_format= tsv
• nextflow_config - Config default value correct: params.kaiju_taxon_rank= species
• nextflow_config - Config default value correct: params.krakenuniq_ram_chunk_size= 16G
• nextflow_config - Config default value correct: params.krakenuniq_batch_size= 20
• nextflow_config - Config default value correct: params.malt_mode= BlastN
• nextflow_config - Config default value correct: params.ganon_report_type= reads
• nextflow_config - Config default value correct: params.ganon_report_toppercentile= 0
• nextflow_config - Config default value correct: params.ganon_report_mincount= 0
• nextflow_config - Config default value correct: params.ganon_report_maxcount= 0
• nextflow_config - Config default value correct: params.standardisation_taxpasta_format= tsv
• nextflow_config - Config default value correct: params.custom_config_version= master
• nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Config default value correct: params.max_cpus= 16
• nextflow_config - Config default value correct: params.max_memory= 128.GB
• nextflow_config - Config default value correct: params.max_time= 240.h
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
• nextflow_config - Config default value correct: params.validate_params= true
• nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
• nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• files_unchanged - CODE_OF_CONDUCT.md matches the template
• files_unchanged - LICENSE matches the template
• files_unchanged - .github/.dockstore.yml matches the template
• files_unchanged - .github/CONTRIBUTING.md matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
• files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
• files_unchanged - .github/workflows/branch.yml matches the template
• files_unchanged - .github/workflows/linting_comment.yml matches the template
• files_unchanged - .github/workflows/linting.yml matches the template
• files_unchanged - assets/email_template.html matches the template
• files_unchanged - assets/email_template.txt matches the template
• files_unchanged - assets/sendmail_template.txt matches the template
• files_unchanged - assets/nf-core-taxprofiler_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-taxprofiler_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-taxprofiler_logo_dark.png matches the template
• files_unchanged - docs/README.md matches the template
• files_unchanged - .gitignore matches the template
• files_unchanged - .prettierignore matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
• readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
• readme - README Zenodo placeholder was replaced with DOI.
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (296 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: awsfulltest.yml
• actions_schema_validation - Workflow validation passed: fix-linting.yml
• actions_schema_validation - Workflow validation passed: branch.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: awstest.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: clean-up.yml
• actions_schema_validation - Workflow validation passed: release-announcements.yml
• actions_schema_validation - Workflow validation passed: download_pipeline.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• multiqc_config - assets/multiqc_config.yml found and not ignored.
• multiqc_config - assets/multiqc_config.yml contains report_section_order
• multiqc_config - assets/multiqc_config.yml contains export_plots
• multiqc_config - assets/multiqc_config.yml contains report_comment
• multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
• multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
• multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
• base_config - conf/base.config found and not ignored.
• base_config - BRACKEN_BRACKEN found in conf/base.config and Nextflow scripts.
• base_config - CENTRIFUGE_KREPORT found in conf/base.config and Nextflow scripts.
• base_config - KRAKENTOOLS_COMBINEKREPORTS_CENTRIFUGE found in conf/base.config and Nextflow scripts.
• base_config - MEGAN_RMA2INFO_TSV found in conf/base.config and Nextflow scripts.
• base_config - MEGAN_RMA2INFO_KRONA found in conf/base.config and Nextflow scripts.
• base_config - FALCO found in conf/base.config and Nextflow scripts.
• modules_config - conf/modules.config found and not ignored.
• modules_config - UNTAR found in conf/modules.config and Nextflow scripts.
• modules_config - FASTQC found in conf/modules.config and Nextflow scripts.
• modules_config - FASTQC_PROCESSED found in conf/modules.config and Nextflow scripts.
• modules_config - FALCO found in conf/modules.config and Nextflow scripts.
• modules_config - FALCO_PROCESSED found in conf/modules.config and Nextflow scripts.
• modules_config - FASTP_SINGLE found in conf/modules.config and Nextflow scripts.
• modules_config - FASTP_PAIRED found in conf/modules.config and Nextflow scripts.
• modules_config - ADAPTERREMOVAL_SINGLE found in conf/modules.config and Nextflow scripts.
• modules_config - ADAPTERREMOVAL_PAIRED found in conf/modules.config and Nextflow scripts.
• modules_config - NONPAREIL_NONPAREIL found in conf/modules.config and Nextflow scripts.
• modules_config - NONPAREIL_CURVE found in conf/modules.config and Nextflow scripts.
• modules_config - NONPAREIL_SET found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_FASTQ found in conf/modules.config and Nextflow scripts.
• modules_config - PORECHOP_PORECHOP found in conf/modules.config and Nextflow scripts.
• modules_config - FILTLONG found in conf/modules.config and Nextflow scripts.
• modules_config - BBMAP_BBDUK found in conf/modules.config and Nextflow scripts.
• modules_config - PRINSEQPLUSPLUS found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - MINIMAP2_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - MINIMAP2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_VIEW found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FASTQ found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_STATS found in conf/modules.config and Nextflow scripts.
• modules_config - MERGE_RUNS found in conf/modules.config and Nextflow scripts.
• modules_config - MALT_RUN found in conf/modules.config and Nextflow scripts.
• modules_config - MEGAN_RMA2INFO_TSV found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKEN2_KRAKEN2 found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKEN2_STANDARD_REPORT found in conf/modules.config and Nextflow scripts.
• modules_config - BRACKEN_BRACKEN found in conf/modules.config and Nextflow scripts.
• modules_config - BRACKEN_COMBINEBRACKENOUTPUTS found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENTOOLS_COMBINEKREPORTS_KRAKEN found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENUNIQ_PRELOADEDKRAKENUNIQ found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENTOOLS_KREPORT2KRONA found in conf/modules.config and Nextflow scripts.
• modules_config - KRONA_CLEANUP found in conf/modules.config and Nextflow scripts.
• modules_config - KRONA_KTIMPORTTEXT found in conf/modules.config and Nextflow scripts.
• modules_config - MEGAN_RMA2INFO_KRONA found in conf/modules.config and Nextflow scripts.
• modules_config - KRONA_KTIMPORTTAXONOMY found in conf/modules.config and Nextflow scripts.
• modules_config - METAPHLAN_METAPHLAN found in conf/modules.config and Nextflow scripts.
• modules_config - METAPHLAN_MERGEMETAPHLANTABLES found in conf/modules.config and Nextflow scripts.
• modules_config - CENTRIFUGE_CENTRIFUGE found in conf/modules.config and Nextflow scripts.
• modules_config - CENTRIFUGE_KREPORT found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENTOOLS_COMBINEKREPORTS_CENTRIFUGE found in conf/modules.config and Nextflow scripts.
• modules_config - KAIJU_KAIJU found in conf/modules.config and Nextflow scripts.
• modules_config - KAIJU_KAIJU2TABLE_SINGLE found in conf/modules.config and Nextflow scripts.
• modules_config - KAIJU_KAIJU2TABLE_COMBINED found in conf/modules.config and Nextflow scripts.
• modules_config - KAIJU_KAIJU2KRONA found in conf/modules.config and Nextflow scripts.
• modules_config - DIAMOND_BLASTX found in conf/modules.config and Nextflow scripts.
• modules_config - MOTUS_PROFILE found in conf/modules.config and Nextflow scripts.
• modules_config - MOTUS_MERGE found in conf/modules.config and Nextflow scripts.
• modules_config - KMCP_SEARCH found in conf/modules.config and Nextflow scripts.
• modules_config - KMCP_PROFILE found in conf/modules.config and Nextflow scripts.
• modules_config - GANON_CLASSIFY found in conf/modules.config and Nextflow scripts.
• modules_config - GANON_REPORT found in conf/modules.config and Nextflow scripts.
• modules_config - GANON_TABLE found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_MERGE found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_STANDARDISE found in conf/modules.config and Nextflow scripts.
• modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
• nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
• nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

• nf-core/tools version 2.14.1
• Run at 2024-07-03 08:41:08

[maxulysse]

@LilyAnderssonLee I think you were a bit overboard.

The idea is to be able to visualize quickly text file and html reports (cf https://github.com/nf-core/rnaseq/blob/dev/tower.yml).
I would remove zip and gzip files.

cf https://docs.seqera.io/platform/23.4.0/reports/overview#limitations

[LilyAnderssonLee]

@LilyAnderssonLee I think you were a bit overboard.

The idea is to be able to visualize quickly text file and html reports (cf https://github.com/nf-core/rnaseq/blob/dev/tower.yml). I would remove zip and gzip files.

cf https://docs.seqera.io/platform/23.4.0/reports/overview#limitations

Ok, I will update it.

[JoseEspinosa]

LGTM

[JoseEspinosa]

Sorry did not see @maxulysse review, please address his suggestions and fix failing test before merging (if they do not run this time correctly, I relaunched it at it seems to me a network issue but not sure)

[jfy133]

@LilyAnderssonLee do you know how fancythe TSV viewer is?

If you can do filtering of tables indeed makes sense to include the taxonomic profiles, but if not I'm not sure the utility of inmcluding those.

HTML reports makes sense, asn does maybe log files, but otherwise...?

[LilyAnderssonLee]

@jfy133 TBH I am not sure about the TSV viewer. I have never run taxprofiler on tower yet. It seems tower reports only support these formats: HTML, csv, tsv, pdf, and txt.

[jfy133]

Apparently it does! (using nf-core/rnaseq on the AWS megatest workspace as an example!)

[jfy133]

I have a feeling some of the tables will fail to render properly as extension is .txt but Mimetype is not set to tSV

But I think we can review this and fix it in the future release, once we have a fulltest run to inspect the output files :)