diff --git a/markdown/developers/create-pipeline.md b/markdown/developers/create-pipeline.md new file mode 100644 index 0000000000..87ef66a840 --- /dev/null +++ b/markdown/developers/create-pipeline.md @@ -0,0 +1,89 @@ +--- +title: Pipeline Creation Instructions +subtitle: Instructions for creating a sanger-tol pipeline +--- + +## nf-core instructions + +This page is heavily inspired by the nf-core page [Adding a new pipeline](https://nf-co.re/docs/contributing/adding_pipelines). + +## Create the pipeline + +All pipelines _must_ use the [nf-core template](https://nf-co.re/docs/contributing/guidelines/requirements/use_the_template). +This is done by using the `nf-core create` command - see [the docs](https://nf-co.re/tools#creating-a-new-pipeline) for detailed instructions. +This tool does lots of things for you: it gives you the correct file structure and boiler plate code +and also sets up the required `git` infrastructure for you to keep your pipeline in sync in the future. + +When asked _Do you want to customize which parts of the template are used ?_, answer `y`. +Then, set the "Pipeline prefix" to "sanger-tol", and when asked to "Skip template areas", +disable "iGenomes config" by: + +- pressing the down arrow to choose the option, +- pressing the space-bar to select it for disablement, +- pressing Enter + +## Push to GitHub + +Create an empty repository on GitHub for your new pipeline under . +Do this by: + +1. going to the GitHub website, +2. clicking `+` then _New Repository_, +3. selecting "sanger-tol" as the Owner. + +Make sure _not_ to initialise it with _any_ file, `README` or `LICENSE`: you just want an empty repository. +You already have these files generated from the nf-core template. + +Leave the repository as "Public". We don't want to hide our pipelines, even when they're in progress. + +Once created, copy the git URL and add this as a remote to your local git repository. +The `nf-core create` command will have initialised a git repository for you, +so all you need to do is add the remote: + +```bash +## Add a remote called 'origin' - this is the default name for a primary remote +git remote add origin https://github.com/sanger-tol/PIPELINE_NAME.git +``` + +The create command also generated the three standard nf-core branches (`master`, `dev` and `TEMPLATE`), +together with an initial commit which is shared between them. +This git structure is required for automatic template synchronisation in the future. + +You first need to rename the `master` branch: + +```bash +git branch -m master main +``` + +Then, you can push these new branches to the remote GitHub repository: + +```bash +git push --all origin +``` + +You should now see the vanilla nf-core template and branches in the github.com web interface. + +## GitHub configuration + +Head up to your repository on GitHub and do the following. + +In the About section on the right, click on the cog wheel and: + +1. Set the URL to . +2. Add the topics `pipeline` and `nextflow`. This is required to enable it on the pipelines website. + - Most pipelines also have `workflow` and `genomics`. +3. Enter a description. + +Then, ask [@muffato](https://github.com/muffato) or [@mcshane](https://github.com/mcshane) to add the repository to: + +1. The ["nextflow_all"](https://github.com/orgs/sanger-tol/teams/nextflow_all) team with the "write" permission +2. The ["nextflow_admin"](https://github.com/orgs/sanger-tol/teams/nextflow_admin) team with the "admin" permission +3. Remove your individual access to the repository + +Finally, ask [@gq1](https://github.com/gq1) to set up the pipeline settings via . + +## Other bits + +The repository needs to be integrated with Zenodo before making the first release. +Better to do it now before anyone forgets ! +Ask [@muffato](https://github.com/muffato) to enable the Zenodo integration. diff --git a/markdown/developers/env.md b/markdown/developers/env.md index 6c97b8afc5..0a0bb7832d 100644 --- a/markdown/developers/env.md +++ b/markdown/developers/env.md @@ -18,7 +18,7 @@ The `main` and `dev` branches are "protected" and can only be modified with pull - Direct push to `main` and `dev` is not allowed, since the two branches are protected. - Within each team (Tree of Life Assembly and Informatics Infrastructure), everyone has ["maintain" access](https://docs.github.com/en/organizations/managing-user-access-to-your-organizations-repositories/managing-repository-roles/repository-roles-for-an-organization#permissions-for-each-role) to the team repositories. - A selection of people from the Informatics Infrastructure team have ["admin" access](https://docs.github.com/en/organizations/managing-user-access-to-your-organizations-repositories/managing-repository-roles/repository-roles-for-an-organization#permissions-for-each-role) to all Nextflow repositories. -- @muffato and @mcshane are ["owners"](https://docs.github.com/en/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization#permissions-for-organization-roles). +- [@muffato](https://github.com/muffato) and [@mcshane](https://github.com/mcshane) are ["owners"](https://docs.github.com/en/organizations/managing-peoples-access-to-your-organization-with-roles/roles-in-an-organization#permissions-for-organization-roles). - Additional contributors can be added, if sponsored by a repository admin. ## Farm environment diff --git a/markdown/developers/pipeline-release.md b/markdown/developers/pipeline-release.md index 2c571b6b7d..6be4ee18ce 100644 --- a/markdown/developers/pipeline-release.md +++ b/markdown/developers/pipeline-release.md @@ -1,6 +1,6 @@ --- -title: Pipeline Release Guidelines -subtitle: Guidelines on releasing a sanger-tol pipeline +title: Pipeline Release Instructions +subtitle: Instructions for releasing a sanger-tol pipeline --- ## nf-core Release Checklist @@ -9,11 +9,26 @@ This page is heavily inspired by the [nf-core release checklist](https://nf-co.r ## Branch model -First a reminder about how we use branches in sanger-tol: +First a reminder about how we use branches in sanger-tol. -- `main` is the release branch. Releases can only happen from the `main` branch and code present on the `main` branch _has_ to be released. -- `dev` is the staging branch. Code on the `dev` branch should always pass tests and be functional (this is a condition of our [review guidelines](/docs/contributing/review_checklist)). Releasing a pipeline involves merging `dev` into `main`. -- Feature branches are where development happens. +`main` is the release branch. Releases can only happen from the `main` branch +and code present on the `main` branch _has_ to be released. `main` can be +updated by merging either the staging branch, `dev`, or a bugfix branch. + +`dev` is the staging branch, which accumulates new features before release. +Code on the `dev` branch should always pass tests and be functional (this is a +condition of our [review guidelines](/docs/contributing/review_checklist)). + +Bugfix branches can be used to patch a released pipeline. The branch needs to +be created off `main` and merged into `main` for immediate release. There needs +to be extra caution when merging a bugfix branch into `main` as there isn't the +`dev` branch as a buffer. + +Our model only supports 1 "active" release at a time. For instance, if you have +already released 1.0.0 and 1.1.0, 1.1.0 is considered the "active" version, and +the next version, according to [Semantic Versioning](https://semver.org/), can +only be 1.1.1, 1.2.0, or 2.0.0. In this example, releasing 1.0.1 is **not** +supported. ## Release steps