Skip to content

Commit

Permalink
change pipeline template and tooling to allow main as default branch
Browse files Browse the repository at this point in the history
  • Loading branch information
mashehu committed Aug 27, 2024
1 parent 65751e9 commit 6fc6da2
Show file tree
Hide file tree
Showing 23 changed files with 78 additions and 61 deletions.
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ These are the most common things requested on pull requests (PRs).
Remember that PRs should be made against the dev branch, unless you're preparing a release.
Learn more about contributing: https://github.com/nf-core/tools/tree/master/.github/CONTRIBUTING.md
Learn more about contributing: https://github.com/nf-core/tools/tree/main/.github/CONTRIBUTING.md
-->

## PR checklist
Expand Down
6 changes: 3 additions & 3 deletions .github/RELEASE_CHECKLIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,11 @@
4. Check that modules/subworkflows in template are up to date with the latest releases
5. Create a PR to `dev` to bump the version in `CHANGELOG.md` and `setup.py` and change the gitpod container to `nfcore/gitpod:latest`.
6. Make sure all CI tests are passing!
7. Create a PR from `dev` to `master`
8. Make sure all CI tests are passing again (additional tests are run on PRs to `master`)
7. Create a PR from `dev` to `main`
8. Make sure all CI tests are passing again (additional tests are run on PRs to `main`)
9. Request review (2 approvals required)
10. Run `rich-codex` to regenerate docs screengrabs (actions `workflow_dispatch` button)
11. Merge the PR into `master`
11. Merge the PR into `main`
12. Wait for CI tests on the commit to passed
13. (Optional but a good idea) Run a manual sync on `nf-core/testpipeline` and check that CI is passing on the resulting PR.
14. Create a new release copying the `CHANGELOG` for that release into the description section.
Expand Down
16 changes: 8 additions & 8 deletions .github/workflows/branch.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
# This workflow is triggered on PRs to main/master branch on the repository
# It fails when someone tries to make a PR against the nf-core `main/master` branch instead of `dev`
on:
pull_request_target:
branches: [master]
branches: [main, master]

jobs:
test:
runs-on: ubuntu-latest
steps:
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
- name: Check PRs
if: github.repository == 'nf-core/tools'
run: |
Expand All @@ -21,7 +21,7 @@ jobs:
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
## This PR is against the `main/master` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
Expand All @@ -31,9 +31,9 @@ jobs:
Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main` branch.
The `main/master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `main/master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
steps:
- name: Check conditions
id: conditions
run: echo "run-tests=${{ github.ref == 'refs/heads/master' || (matrix.runner == 'ubuntu-20.04' && matrix.python-version == '3.8') }}" >> "$GITHUB_OUTPUT"
run: echo "run-tests=${{ github.ref == 'refs/heads/master' || github.ref == 'refs/heads/main' || (matrix.runner == 'ubuntu-20.04' && matrix.python-version == '3.8') }}" >> "$GITHUB_OUTPUT"

outputs:
python-version: ${{ matrix.python-version }}
Expand Down
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
<h1>
<picture>
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/nf-core/tools/master/docs/images/nfcore-tools_logo_dark.png">
<img alt="nf-core/tools" src="https://raw.githubusercontent.com/nf-core/tools/master/docs/images/nfcore-tools_logo_light.png">
<source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/nf-core/tools/main/docs/images/nfcore-tools_logo_dark.png">
<img alt="nf-core/tools" src="https://raw.githubusercontent.com/nf-core/tools/main/docs/images/nfcore-tools_logo_light.png">
</picture>
</h1><!-- omit in toc -->

[![Python tests](https://github.com/nf-core/tools/workflows/Python%20tests/badge.svg?branch=master&event=push)](https://github.com/nf-core/tools/actions?query=workflow%3A%22Python+tests%22+branch%3Amaster)
[![codecov](https://codecov.io/gh/nf-core/tools/branch/master/graph/badge.svg)](https://codecov.io/gh/nf-core/tools)
[![Python tests](https://github.com/nf-core/tools/workflows/Python%20tests/badge.svg?branch=main&event=push)](https://github.com/nf-core/tools/actions?query=workflow%3A%22Python+tests%22+branch%3Amain)
[![codecov](https://codecov.io/gh/nf-core/tools/branch/main/graph/badge.svg)](https://codecov.io/gh/nf-core/tools)
[![code style: prettier](https://img.shields.io/badge/code%20style-prettier-ff69b4.svg)](https://github.com/prettier/prettier)
[![code style: Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v1.json)](https://github.com/charliermarsh/ruff)

Expand Down
5 changes: 3 additions & 2 deletions nf_core/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import logging
import os
import sys
from pathlib import Path

import rich
import rich.console
Expand Down Expand Up @@ -286,7 +287,7 @@ def command_pipelines_create(ctx, name, description, author, version, force, out
@click.option(
"--release",
is_flag=True,
default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master"
default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"]
and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/")
and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools",
help="Execute additional checks for release-ready workflows.",
Expand Down Expand Up @@ -2139,7 +2140,7 @@ def command_download(
@click.option(
"--release",
is_flag=True,
default=os.path.basename(os.path.dirname(os.environ.get("GITHUB_REF", "").strip(" '\""))) == "master"
default=Path(os.environ.get("GITHUB_REF", "").strip(" '\"")).parent.name in ["master", "main"]
and os.environ.get("GITHUB_REPOSITORY", "").startswith("nf-core/")
and not os.environ.get("GITHUB_REPOSITORY", "") == "nf-core/tools",
help="Execute additional checks for release-ready workflows.",
Expand Down
4 changes: 2 additions & 2 deletions nf_core/components/create.py
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,7 @@ def _get_module_structure_components(self):
if self.process_label is None:
log.info(
"Provide an appropriate resource label for the process, taken from the "
"[link=https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config#L29]nf-core pipeline template[/link].\n"
"[link=https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config#L29]nf-core pipeline template[/link].\n"
"For example: {}".format(", ".join(process_label_defaults))
)
while self.process_label is None:
Expand All @@ -260,7 +260,7 @@ def _get_module_structure_components(self):
"Where applicable all sample-specific information e.g. 'id', 'single_end', 'read_group' "
"MUST be provided as an input via a Groovy Map called 'meta'. "
"This information may [italic]not[/] be required in some instances, for example "
"[link=https://github.com/nf-core/modules/blob/master/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]."
"[link=https://github.com/nf-core/modules/blob/main/modules/nf-core/bwa/index/main.nf]indexing reference genome files[/link]."
)
while self.has_meta is None:
self.has_meta = rich.prompt.Confirm.ask(
Expand Down
6 changes: 3 additions & 3 deletions nf_core/pipeline-template/.github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,9 +59,9 @@ These tests are run both with the latest available version of `Nextflow` and als

:warning: Only in the unlikely and regretful event of a release happening with a bug.

- On your own fork, make a new branch `patch` based on `upstream/master`.
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- A PR should be made on `master` from patch to directly this particular bug.
- A PR should be made on `main`/`master` from patch to directly this particular bug.

{% if is_nfcore -%}

Expand Down Expand Up @@ -100,7 +100,7 @@ Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

Expand Down
4 changes: 2 additions & 2 deletions nf_core/pipeline-template/.github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,14 @@ These are the most common things requested on pull requests (PRs).
Remember that PRs should be made against the dev branch, unless you're preparing a pipeline release.
Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md)
Learn more about contributing: [CONTRIBUTING.md](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md)
-->

## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/master/.github/CONTRIBUTING.md)
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/{{ name }}/tree/{{ default_branch }}/.github/CONTRIBUTING.md)
{%- if is_nfcore %}
- [ ] If necessary, also make a PR on the {{ name }} _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
{%- endif %}
Expand Down
3 changes: 2 additions & 1 deletion nf_core/pipeline-template/.github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
name: nf-core AWS full size tests
# This workflow is triggered on PRs opened against the master branch.
# This workflow is triggered on PRs opened against the main/master branch.
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
# It runs the -profile 'test_full' on AWS batch

on:
pull_request:
branches:
- main
- master
workflow_dispatch:
pull_request_review:
Expand Down
18 changes: 10 additions & 8 deletions nf_core/pipeline-template/.github/workflows/branch.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,17 @@
name: nf-core branch protection
# This workflow is triggered on PRs to master branch on the repository
# It fails when someone tries to make a PR against the nf-core `master` branch instead of `dev`
# This workflow is triggered on PRs to main/master branch on the repository
# It fails when someone tries to make a PR against the nf-core `main/master` branch instead of `dev`
on:
pull_request_target:
branches: [master]
branches:
- main
- master

jobs:
test:
runs-on: ubuntu-latest
steps:
# PRs to the nf-core repo master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
# PRs to the nf-core repo main/master branch are only ok if coming from the nf-core repo `dev` or any `patch` branches
- name: Check PRs
if: github.repository == '{{ name }}'
run: |
Expand All @@ -22,7 +24,7 @@ jobs:
uses: mshick/add-pr-comment@b8f338c590a895d50bcbfa6c5859251edc8952fc # v2
with:
message: |
## This PR is against the `master` branch :x:
## This PR is against the `main/master` branch :x:
* Do not close this PR
* Click _Edit_ and change the `base` to `dev`
Expand All @@ -32,9 +34,9 @@ jobs:
Hi @${{ github.event.pull_request.user.login }},
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `master` branch.
The `master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
It looks like this pull-request is has been made against the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `main/master` branch.
The `main/master` branch on nf-core repositories should always contain code from the latest release.
Because of this, PRs to `main/master` are only allowed if they come from the [${{github.event.pull_request.head.repo.full_name }}](https://github.com/${{github.event.pull_request.head.repo.full_name }}) `dev` branch.
You do not need to close this PR, you can change the target branch to `dev` by clicking the _"Edit"_ button at the top of this page.
Note that even after this, the test will continue to show as failing until you push a new commit.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Test successful pipeline download with 'nf-core pipelines download'

# Run the workflow when:
# - dispatched manually
# - when a PR is opened or reopened to master branch
# - when a PR is opened or reopened to main/master branch
# - the head branch of the pull request is updated, i.e. if fixes for a release are pushed last minute to dev.
on:
workflow_dispatch:
Expand All @@ -17,9 +17,11 @@ on:
- edited
- synchronize
branches:
- main
- master
pull_request_target:
branches:
- main
- master

env:
Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ An extensive list of references for the tools used by the pipeline can be found
You can cite the `nf-core` publication as follows:

{% else -%}
This pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/master/LICENSE).
This pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/{{ default_branch }}/LICENSE).

{% endif -%}

Expand Down
2 changes: 1 addition & 1 deletion nf_core/pipeline-template/assets/schema_input.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"$schema": "http://json-schema.org/draft-07/schema",
"$id": "https://raw.githubusercontent.com/{{ name }}/master/assets/schema_input.json",
"$id": "https://raw.githubusercontent.com/{{ name }}/{{ default_branch }}/assets/schema_input.json",
"title": "{{ name }} pipeline - params.input schema",
"description": "Schema for the file provided with params.input",
"type": "array",
Expand Down
Loading

0 comments on commit 6fc6da2

Please sign in to comment.