Merge pull request #59 from nf-core/dev

Dev -> Master for 1.5 release

.gitattributes

@@ -1 +1,3 @@
 *.config linguist-language=nextflow
+modules/nf-core/** linguist-generated
+subworkflows/nf-core/** linguist-generated

.github/CONTRIBUTING.md

@@ -61,16 +61,20 @@ For further information/help, please consult the [nf-core/fetchngs documentation
 
 To make the nf-core/fetchngs code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
 
-### Adding a new step or module
+### Adding a new step
 
-If you wish to contribute a new step or module please see the [official guidelines](https://nf-co.re/developers/adding_modules#new-module-guidelines-and-pr-review-checklist) and use the following coding standards:
+If you wish to contribute a new step, please use the following coding standards:
 
-1. Add any new flags/options to `nextflow.config` with a default (see section below).
-2. Add any new flags/options to `nextflow_schema.json` with help text via `nf-core schema build`.
-3. Add sanity checks for all relevant parameters.
-4. Perform local tests to validate that the new code works as expected.
-5. If applicable, add a new test command in `.github/workflow/ci.yml`.
-6. Add any descriptions of output files to `docs/output.md`.
+1. Define the corresponding input channel into your new process from the expected previous process channel
+2. Write the process block (see below).
+3. Define the output channel if needed (see below).
+4. Add any new parameters to `nextflow.config` with a default (see below).
+5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
+6. Add sanity checks and validation for all relevant parameters.
+7. Perform local tests to validate that the new code works as expected.
+8. If applicable, add a new test command in `.github/workflow/ci.yml`.
+9. Update MultiQC config `assets/multiqc_config.yaml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
+10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.
 
 ### Default values
 
@@ -82,12 +86,14 @@ Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
 
 Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
 
-### Channel naming convention
+The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
+
+### Naming schemes
 
 Please use the following naming schemes, to make it easy to understand what is going where.
 
-* Initial process channel: `ch_output_from_<process>`
-* Intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
+* initial process channel: `ch_output_from_<process>`
+* intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
 
 ### Nextflow version bumping
 

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,52 @@
+
+name: Bug report
+description: Report something that is broken or incorrect
+labels: bug
+body:
+
+  - type: markdown
+    attributes:
+      value: |
+        Before you post this issue, please check the documentation:
+
+        - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
+        - [nf-core/fetchngs pipeline documentation](https://nf-co.re/fetchngs/usage)
+
+  - type: textarea
+    id: description
+    attributes:
+      label: Description of the bug
+      description: A clear and concise description of what the bug is.
+    validations:
+      required: true
+
+  - type: textarea
+    id: command_used
+    attributes:
+      label: Command used and terminal output
+      description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
+      render: console
+      placeholder: |
+        $ nextflow run ...
+
+        Some output where something broke
+
+  - type: textarea
+    id: files
+    attributes:
+      label: Relevant files
+      description: |
+        Please upload (drag and drop) and relevant files. Make into a `.zip` file if the extension is not allowed.
+        Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.
+
+  - type: textarea
+    id: system
+    attributes:
+      label: System information
+      description: |
+        * Nextflow version _(eg. 21.10.3)_
+        * Hardware _(eg. HPC, Desktop, Cloud)_
+        * Executor _(eg. slurm, local, awsbatch)_
+        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
+        * OS _(eg. CentOS Linux, macOS, Linux Mint)_
+        * Version of nf-core/fetchngs _(eg. 1.1, 1.5, 1.8.2)_

.github/ISSUE_TEMPLATE/config.yml

@@ -1,4 +1,3 @@
-blank_issues_enabled: false
 contact_links:
   - name: Join nf-core
     url: https://nf-co.re/join

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,11 @@
+name: Feature request
+description: Suggest an idea for the nf-core/fetchngs pipeline
+labels: enhancement
+body:
+  - type: textarea
+    id: description
+    attributes:
+      label: Description of feature
+      description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
+    validations:
+      required: true

.github/workflows/awsfulltest.yml

@@ -14,10 +14,11 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Launch workflow via tower
-        uses: nf-core/tower-action@master
+        uses: nf-core/tower-action@v2
+
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
-          bearer_token: ${{ secrets.TOWER_BEARER_TOKEN }}
+          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
           pipeline: ${{ github.repository }}
           revision: ${{ github.sha }}
@@ -26,4 +27,6 @@ jobs:
             {
               "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/fetchngs/results-${{ github.sha }}"
             }
-          profiles: '[ "test_full", "aws_tower" ]'
+          profiles: test_full,aws_tower
+          pre_run_script: 'export NXF_VER=21.10.3'
+

.github/workflows/awstest.yml

@@ -11,11 +11,11 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Launch workflow via tower
-        uses: nf-core/tower-action@master
+        uses: nf-core/tower-action@v2
 
         with:
           workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
-          bearer_token: ${{ secrets.TOWER_BEARER_TOKEN }}
+          access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
           compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
           pipeline: ${{ github.repository }}
           revision: ${{ github.sha }}
@@ -24,4 +24,6 @@ jobs:
             {
               "outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/fetchngs/results-${{ github.sha }}"
             }
-          profiles: '[ "test", "aws_tower" ]'
+          profiles: test,aws_tower
+          pre_run_script: 'export NXF_VER=21.10.3'
+

.github/workflows/ci.yml

@@ -8,9 +8,6 @@ on:
   release:
     types: [published]
 
-# Uncomment if we need an edge release of Nextflow again
-# env: NXF_EDGE: 1
-
 jobs:
   test:
     name: Run workflow tests
@@ -22,7 +19,9 @@ jobs:
     strategy:
       matrix:
         # Nextflow versions: check pipeline minimum and latest edge version
-        nxf_ver: ["NXF_VER=21.04.0", "NXF_EDGE=1"]
+        nxf_ver:
+          - 'NXF_VER=21.10.3'
+        #  - 'NXF_EDGE=1'
     steps:
       - name: Check out pipeline code
         uses: actions/checkout@v2

.github/workflows/linting_comment.yml

@@ -15,6 +15,7 @@ jobs:
         uses: dawidd6/action-download-artifact@v2
         with:
           workflow: linting.yml
+          workflow_conclusion: completed
 
       - name: Get PR number
         id: pr_number

.nf-core.yml

@@ -1,10 +1,15 @@
 lint:
   files_unchanged:
+    - .gitattributes
+    - .github/ISSUE_TEMPLATE/config.yml
+    - .github/workflows/linting_comment.yml
     - .github/CONTRIBUTING.md
     - assets/sendmail_template.txt
     - lib/NfcoreSchema.groovy
     - lib/NfcoreTemplate.groovy
   files_exist:
+    - .github/ISSUE_TEMPLATE/bug_report.md
+    - .github/ISSUE_TEMPLATE/feature_request.md
     - bin/scrape_software_versions.py
     - modules/local/get_software_versions.nf
   actions_ci: False

CHANGELOG.md

@@ -3,6 +3,14 @@
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [[1.5](https://github.com/nf-core/fetchngs/releases/tag/1.5)] - 2021-12-01
+
+* Finish porting the pipeline to the updated Nextflow DSL2 syntax adopted on nf-core/modules
+    * Bump minimum Nextflow version from `21.04.0` -> `21.10.3`
+    * Removed `--publish_dir_mode` as it is no longer required for the new syntax
+
+### Enhancements & fixes
+
 ## [[1.4](https://github.com/nf-core/fetchngs/releases/tag/1.4)] - 2021-11-09
 
 ### Enhancements & fixes

README.md

@@ -5,7 +5,7 @@
 [![AWS CI](https://img.shields.io/badge/CI%20tests-full%20size-FF9900?labelColor=000000&logo=Amazon%20AWS)](https://nf-co.re/fetchngs/results)
 [![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.5070524-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.5070524)
 
-[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A521.04.0-23aa62.svg?labelColor=000000)](https://www.nextflow.io/)
+[![Nextflow](https://img.shields.io/badge/nextflow%20DSL2-%E2%89%A521.10.3-23aa62.svg?labelColor=000000)](https://www.nextflow.io/)
 [![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)
 [![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)
 [![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)
@@ -48,7 +48,7 @@ The columns in the auto-created samplesheet can be tailored to be accepted out-o
 
 ## Quick Start
 
-1. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`>=21.04.0`)
+1. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`>=21.10.3`)
 
 2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/), [`Podman`](https://podman.io/), [`Shifter`](https://nersc.gitlab.io/development/shifter/how-to-use/) or [`Charliecloud`](https://hpc.github.io/charliecloud/) for full pipeline reproducibility _(please only use [`Conda`](https://conda.io/miniconda.html) as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_