Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dsl2 port #199

Merged
merged 191 commits into from
Nov 3, 2022
Merged
Show file tree
Hide file tree
Changes from 136 commits
Commits
Show all changes
191 commits
Select commit Hold shift + click to select a range
0625da5
check in required nf-core modules
phue Jan 19, 2021
a666c5d
add required bismark modules
phue Jan 19, 2021
b71b35a
add required bwameth modules
phue Jan 19, 2021
78b492a
add required methyldackel modules
phue Jan 19, 2021
470b74e
add required qualimap/bamqc module
phue Jan 19, 2021
8fa9177
add required samtools/faidx module
phue Jan 19, 2021
aa84f29
add functions from nf-core/tools dsl2 template
phue Jan 19, 2021
0cbc1c2
add samplesheet functionality
phue Jan 19, 2021
7dc73ff
add bismark subworkflow
phue Jan 19, 2021
811c806
add bwameth subworkflow
phue Jan 19, 2021
f31c396
wire up subworkflows
phue Jan 19, 2021
ed6e252
add file placeholder
phue Jan 19, 2021
30bd4f3
update test.config to use samplesheet
phue Jan 19, 2021
2c7eda9
update base.config
phue Jan 19, 2021
4152bae
update igenomes.config
phue Jan 19, 2021
9dc0108
sync some changes from dsl2 template
phue Jan 19, 2021
e4abecd
bump version to 2.0dev
phue Jan 19, 2021
f47037b
check in required nf-core modules
phue Jan 19, 2021
472bc02
add required bismark modules
phue Jan 19, 2021
186da26
add required bwameth modules
phue Jan 19, 2021
1f76e0d
add required methyldackel modules
phue Jan 19, 2021
51e7518
add required qualimap/bamqc module
phue Jan 19, 2021
1a9d597
add required samtools/faidx module
phue Jan 19, 2021
baf4faa
add functions from nf-core/tools dsl2 template
phue Jan 19, 2021
eaf7ad1
add samplesheet functionality
phue Jan 19, 2021
3994b73
add bismark subworkflow
phue Jan 19, 2021
d6cb697
add bwameth subworkflow
phue Jan 19, 2021
e544a1a
Merge dev
phue Mar 22, 2021
70882f2
add file placeholder
phue Jan 19, 2021
9ebf9ac
update test.config to use samplesheet
phue Jan 19, 2021
25b7f65
update base.config
phue Jan 19, 2021
2aa04a5
sync some changes from dsl2 template
phue Jan 19, 2021
bbf3d28
bump version to 2.0dev
phue Jan 19, 2021
0eb6e9f
update test profile for dsl2
phue Mar 22, 2021
9be06d4
remove files that become obsolete with dsl2
phue Mar 22, 2021
e83dddc
Merge branch 'dsl2' of https://github.com/phue/methylseq into dsl2
phue Mar 22, 2021
655d938
update ci.yml
phue Mar 22, 2021
f1df433
update nextflow.config
phue Mar 22, 2021
a88c631
prepare for sync with nf-core/modules
phue Mar 23, 2021
99c3768
Template update for nf-core/tools version 1.14.dev0
phue Mar 23, 2021
8eb68bc
Merge branch 'TEMPLATE' into dsl2
phue Mar 23, 2021
0243d13
sync with nf-core/modules
phue Mar 23, 2021
867b4ee
cleanup some duplicated files
phue Mar 24, 2021
7b3c04a
finish restructuring; create local modules where needed
phue Mar 24, 2021
34305ef
update nextflow_schema.json
phue Mar 24, 2021
9917081
enable tests for dsl2 branch
phue Mar 25, 2021
efbf6ef
fix conda and podman profiles
phue Mar 29, 2021
26560d9
remove duplicated section in README
phue Mar 29, 2021
df97fa1
fix bismark/align argument parsing
phue Mar 29, 2021
16383a8
improve handling of iGenomes indices
phue Mar 29, 2021
5ad41fe
bismark workflow: only publish sorted bam file
phue Mar 29, 2021
aae268f
bismark/report: allow skipping of read deduplication
phue Apr 3, 2021
992b25b
bismark/summary improve matching of process output
phue Apr 3, 2021
9ec4849
methyldackel: fix version scraping
phue Apr 3, 2021
0fd1614
fix issue leading to missing data in multiqc report
phue Apr 3, 2021
dbaaaa1
Merge remote-tracking branch 'origin/dev' into dsl2
phue Apr 4, 2021
a17f668
minor cleanup
phue Apr 4, 2021
aff48f4
cleanup groovy libs
phue Apr 4, 2021
44f0e14
bump bwameth container build to match conda version
phue Apr 4, 2021
cf3f3b2
lint with nf 21.03.0-edge
phue Apr 6, 2021
f651839
update .nf-core-lint.yaml
phue Apr 6, 2021
93aedbe
update .nf-core-lint.yaml
phue Apr 6, 2021
c8aa313
update test profiles
phue Apr 8, 2021
c43e8c9
Manually fix merge conflicts
ewels Jun 24, 2021
f6b18b3
Bring bwameth fix from DSL1 into DSL2 module
ewels Jun 24, 2021
06b5935
Try removing some of the nf-core linting ignores
ewels Jun 24, 2021
294e8a8
Don't pin Nextflow version in linting.yml
ewels Jun 24, 2021
9dc98c9
Update lib/NfcoreSchema.groovy to template version, update .nf-core-l…
ewels Jun 24, 2021
d729f18
Template update for nf-core/tools version 2.0
KevinMenden Jul 13, 2021
5653875
Template update for nf-core/tools version 2.0.1
nf-core-bot Jul 13, 2021
b28f2cd
Template update for nf-core/tools version 2.1
nf-core-bot Jul 27, 2021
df69589
Update .nf-core-lint.yaml
ewels Nov 4, 2021
83afd2a
Template update for nf-core/tools version 2.2
nf-core-bot Dec 14, 2021
837a460
Template update for nf-core/tools version 2.3
nf-core-bot Mar 15, 2022
829c0d6
Template update for nf-core/tools version 2.3.dev0
Mar 16, 2022
d539afb
Merge branch 'TEMPLATE' into dsl2; rough commit
njspix Mar 17, 2022
3a36653
update modules
njspix Mar 17, 2022
63a3cc7
more minor tweaks to get lint at least running
njspix Mar 17, 2022
79241c8
CLI updates
Mar 17, 2022
a03c652
remove duplicates in schema
njspix Mar 17, 2022
fdf07f5
update check_samplesheet to make genome optional
njspix Mar 18, 2022
406e12c
more updates for optional --genome
njspix Mar 18, 2022
6204858
make --genome or --fasta input optional
njspix Mar 18, 2022
45dd439
Template update for nf-core/tools version 2.3.1
nf-core-bot Mar 23, 2022
3bab67d
Template update for nf-core/tools version 2.3.2
nf-core-bot Mar 24, 2022
30cf1e1
trying to get new plumbing working
njspix Mar 31, 2022
005d901
got plumbing working through fastqc (!)
njspix Mar 31, 2022
9d18e0a
try to get executable bits right
Apr 1, 2022
8bae211
Template update for nf-core/tools version 2.3.2
Apr 1, 2022
92568c2
merge latest template; revert samplesheet py
Apr 1, 2022
029280e
update modules
njspix Apr 1, 2022
b0316b5
alignment plumbing seems ok
njspix Apr 1, 2022
95c4335
plumbing seems to be working all the way through
njspix Apr 1, 2022
436780a
put fasta back in test config
njspix Apr 1, 2022
191ee66
prettier
Apr 4, 2022
0f3748c
try to fix a few lint errors
njspix Apr 4, 2022
dec1c1f
more updates...
njspix Apr 7, 2022
ab4e061
plumbing seems to be working for bwameth now
njspix Apr 7, 2022
9d41646
formatting
njspix Apr 8, 2022
440c8e8
fix editorconfig
njspix Apr 25, 2022
7694bf3
fix linting but broke ci tests
njspix Apr 25, 2022
b71bad6
prettier
Apr 25, 2022
19cce3a
update modules
njspix Apr 26, 2022
9c50c93
run prettier
njspix Apr 26, 2022
09eaee7
add test_paired
njspix Apr 26, 2022
d2cf769
clean up 'no process matching config' warnings
njspix Apr 26, 2022
c85560d
remove unused modules
njspix Apr 26, 2022
5378bc6
Merge pull request #230 from njspix/dsl2
njspix Apr 26, 2022
7586942
try to publish bams
njspix May 10, 2022
52e1f1c
publishDir changes
njspix May 10, 2022
632b5e2
more publish fixes
njspix May 10, 2022
acdc4af
another publishdir fix
njspix May 10, 2022
dd24336
Template update for nf-core/tools version 2.4
nf-core-bot May 16, 2022
065d744
few more publish alterations
njspix May 18, 2022
b2c4541
update modules
njspix May 18, 2022
41f2f64
fix a few linting errors
njspix May 18, 2022
4770b8d
run prettier
njspix May 18, 2022
ec2aba6
update picard_markdup to long syntax
njspix May 18, 2022
bff43a3
lint
njspix May 19, 2022
5305a52
Template update for nf-core/tools version 2.4.1
May 19, 2022
35cd0f3
Merge branch 'TEMPLATE' into dsl2
njspix May 19, 2022
316f217
add gcref params for qualimap
njspix May 19, 2022
402182d
change plumbing for bismark to sort before dedup
njspix May 19, 2022
78e6826
change bismark output channel
njspix May 19, 2022
0f379f0
update config andschema
njspix May 19, 2022
7be2ef5
preemptive change to bismark
njspix May 19, 2022
22ce600
syntax and prettier
njspix May 19, 2022
ee420ea
make schema lint happy
njspix May 19, 2022
1e00c59
add fasta param back in
njspix May 19, 2022
8346941
fix config problems
njspix May 19, 2022
b468f7c
add enable_conda to schema
njspix May 19, 2022
c9c40a7
prettier
May 19, 2022
2798219
revert to nf-core bismark/methylationextractor
May 20, 2022
08d6448
bismark/methextract pe-specific args in config
May 20, 2022
8812fae
closure for querying meta in config
May 20, 2022
ba50ea9
remove flag from test profile
njspix May 20, 2022
dbacb13
remove functions.nf
njspix May 25, 2022
35c68c2
allow specify genome with --genome
njspix Jun 27, 2022
e9cb25a
Revert "change plumbing for bismark to sort before dedup"
njspix Jun 28, 2022
23b7a98
fix plumbing for sorting
njspix Jun 28, 2022
b23db7c
add nextseq trimming option
njspix Jul 1, 2022
9f966f9
add nextseq_trim option and update schema
njspix Jul 1, 2022
745bf9a
change nextseq_trim defaults
njspix Jul 7, 2022
10af102
Template update for nf-core/tools version 2.5
nf-core-bot Aug 30, 2022
9384532
Template update for nf-core/tools version 2.5.1
nf-core-bot Sep 1, 2022
79c8341
Merge branch 'nf-core-template-merge-2.5.1' into dsl2
edmundmiller Sep 14, 2022
d27ddd4
Fix CI workflows
ewels Sep 14, 2022
d9ea338
Remove .vscode
ewels Sep 14, 2022
eb93ad5
Merge pull request #244 from ewels/dsl2
ewels Sep 14, 2022
1fe73a4
Update modules.json to new format
ewels Sep 14, 2022
b145472
Updated all modules to latest versions
ewels Sep 14, 2022
19ae986
Removed unused get_software_versions.nf local module
ewels Sep 14, 2022
7083346
Tidy up CITATIONS.md
ewels Sep 14, 2022
1903308
Add pipeline Zenodo DOI to lib and citations.
ewels Sep 14, 2022
105936f
Remove old lib files
ewels Sep 14, 2022
9339084
Update MultiQC and fix channels
ewels Sep 15, 2022
c040510
Make nextflow.config match the template as much as possible
ewels Sep 15, 2022
8232a44
Merge pull request #245 from ewels/dsl2
ewels Sep 16, 2022
91d343a
Merge pull request #246 from ewels/dsl2_remove_lib
ewels Sep 27, 2022
34b061e
Update modules
ewels Sep 28, 2022
016bf45
Add label and when condition to samplesheet_check local module
ewels Sep 28, 2022
4e36533
Merge pull request #247 from ewels/dsl2
ewels Sep 28, 2022
4fb9262
Template update for nf-core/tools version 2.6
nf-core-bot Oct 4, 2022
4a127bc
Update modules.json to new format
ewels Nov 2, 2022
c7c2664
Moved modules up a directory for restructuring
ewels Nov 2, 2022
d15d1f6
Update modules import paths
ewels Nov 2, 2022
b64cd98
Reset accidentally broken modules.json
ewels Nov 2, 2022
406998b
Update modules
ewels Nov 2, 2022
07be643
Manually update nextflow config, schema and workflow for template update
ewels Nov 2, 2022
02e5f1c
Tools v2.6 template update - fix merge conflicts
ewels Nov 2, 2022
2ed45d9
Remove duplication of ch_multiqc_custom_config
ewels Nov 2, 2022
73136bb
Add fasta and fai to Picard markdups
ewels Nov 2, 2022
d71b9be
Remove unused BISMARK_INDEX modules config
ewels Nov 2, 2022
2b6222b
Add fasta_index, bismark_index and bwa_meth_index params back again
ewels Nov 2, 2022
dc762a0
Couple of code comments
ewels Nov 2, 2022
98553d2
Ditch ch_ prefixes
ewels Nov 2, 2022
9314ba9
Remove methods description TODO prompt
ewels Nov 2, 2022
15ec8f9
Alias both samtools_sort processes with different names
ewels Nov 2, 2022
cbae9aa
Add the reference genome index path back to the CI
ewels Nov 2, 2022
b077b4c
Use file() on supplied reference paths
ewels Nov 2, 2022
6702f92
Refactor version collection in subworkflows
ewels Nov 2, 2022
826c59c
Emit versions, not ch_versions
ewels Nov 2, 2022
e2a3eec
Ok use versions not ch_versions EVERYWHERE
ewels Nov 2, 2022
3ac7cd4
Update CI: BWA-meth takes index as a directory now, not a stub path.
ewels Nov 2, 2022
95c0929
Correct paths to previously generated indices in CI tests
ewels Nov 3, 2022
ee781fa
Need to go to bed..
ewels Nov 3, 2022
97501c3
Merge pull request #249 from ewels/dsl2
ewels Nov 3, 2022
0a76d44
Update methylkit/extract module.
ewels Nov 3, 2022
68d2454
Add some extra pipeline flags on the second CI test run
ewels Nov 3, 2022
4b750a3
Merge pull request #250 from ewels/dsl2
ewels Nov 3, 2022
3785c0b
CI tests: Use -resume to speed things along
ewels Nov 3, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
indent_size = 4
indent_style = space

[*.{md,yml,yaml,html,css,scss,js}]
indent_size = 2

# These files are edited and tested upstream in nf-core/modules
[/modules/nf-core/**]
charset = unset
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset
indent_size = unset

[/assets/email*]
indent_size = unset
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
*.config linguist-language=nextflow
modules/nf-core/** linguist-generated
subworkflows/nf-core/** linguist-generated
55 changes: 15 additions & 40 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@ Contributions to the code are even more welcome ;)

If you'd like to write some code for nf-core/methylseq, the standard workflow is as follows:

1. Check that there isn't already an issue about your idea in the [nf-core/methylseq issues](https://github.com/nf-core/methylseq/issues) to avoid duplicating work
* If there isn't one already, please create one so that others know you're working on this
1. Check that there isn't already an issue about your idea in the [nf-core/methylseq issues](https://github.com/nf-core/methylseq/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/methylseq repository](https://github.com/nf-core/methylseq) to your GitHub account
3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
4. Use `nf-core schema build .` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged

If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).
Expand Down Expand Up @@ -49,9 +48,9 @@ These tests are run both with the latest available version of `Nextflow` and als

:warning: Only in the unlikely and regretful event of a release happening with a bug.

* On your own fork, make a new branch `patch` based on `upstream/master`.
* Fix the bug, and bump version (X.Y.Z+1).
* A PR should be made on `master` from patch to directly this particular bug.
- On your own fork, make a new branch `patch` based on `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- A PR should be made on `master` from patch to directly this particular bug.

## Getting help

Expand All @@ -68,22 +67,19 @@ If you wish to contribute a new step, please use the following coding standards:
1. Define the corresponding input channel into your new process from the expected previous process channel
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new flags/options to `nextflow.config` with a default (see below).
5. Add any new flags/options to `nextflow_schema.json` with help text (with `nf-core schema build .`).
6. Add any new flags/options to the help message (for integer/text parameters, print to help the corresponding `nextflow.config` parameter).
7. Add sanity checks for all relevant parameters.
8. Add any new software to the `scrape_software_versions.py` script in `bin/` and the version command to the `scrape_software_versions` process in `main.nf`.
9. Do local tests that the new code works properly and as expected.
10. Add a new test command in `.github/workflow/ci.yaml`.
11. If applicable add a [MultiQC](https://https://multiqc.info/) module.
12. Update MultiQC config `assets/multiqc_config.yaml` so relevant suffixes, name clean up, General Statistics Table column order, and module figures are in the right order.
13. Optional: Add any descriptions of MultiQC report sections and output files to `docs/output.md`.
4. Add any new parameters to `nextflow.config` with a default (see below).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
6. Add sanity checks and validation for all relevant parameters.
7. Perform local tests to validate that the new code works as expected.
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module.
10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`.

### Default values

Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.

Once there, use `nf-core schema build .` to add to `nextflow_schema.json`.
Once there, use `nf-core schema build` to add to `nextflow_schema.json`.

### Default processes resource requirements

Expand All @@ -95,34 +91,13 @@ The process resources can be passed on to the tool dynamically within the proces

Please use the following naming schemes, to make it easy to understand what is going where.

* initial process channel: `ch_output_from_<process>`
* intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`
- initial process channel: `ch_output_from_<process>`
- intermediate and terminal channels: `ch_<previousprocess>_for_<nextprocess>`

### Nextflow version bumping

If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`

### Software version reporting

If you add a new tool to the pipeline, please ensure you add the information of the tool to the `get_software_version` process.

Add to the script block of the process, something like the following:

```bash
<YOUR_TOOL> --version &> v_<YOUR_TOOL>.txt 2>&1 || true
```

or

```bash
<YOUR_TOOL> --help | head -n 1 &> v_<YOUR_TOOL>.txt 2>&1 || true
```

You then need to edit the script `bin/scrape_software_versions.py` to:

1. Add a Python regex for your tool's `--version` output (as in stored in the `v_<YOUR_TOOL>.txt` file), to ensure the version is reported as a `v` and the version number e.g. `v2.1.1`
2. Add a HTML entry to the `OrderedDict` for formatting in MultiQC.

### Images and figures

For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines).
64 changes: 0 additions & 64 deletions .github/ISSUE_TEMPLATE/bug_report.md

This file was deleted.

50 changes: 50 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name: Bug report
description: Report something that is broken or incorrect
labels: bug
body:
- type: markdown
attributes:
value: |
Before you post this issue, please check the documentation:

- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
- [nf-core/methylseq pipeline documentation](https://nf-co.re/methylseq/usage)

- type: textarea
id: description
attributes:
label: Description of the bug
description: A clear and concise description of what the bug is.
validations:
required: true

- type: textarea
id: command_used
attributes:
label: Command used and terminal output
description: Steps to reproduce the behaviour. Please paste the command you used to launch the pipeline and the output from your terminal.
render: console
placeholder: |
$ nextflow run ...

Some output where something broke

- type: textarea
id: files
attributes:
label: Relevant files
description: |
Please drag and drop the relevant files here. Create a `.zip` archive if the extension is not allowed.
Your verbose log file `.nextflow.log` is often useful _(this is a hidden file in the directory where you launched the pipeline)_ as well as custom Nextflow configuration files.

- type: textarea
id: system
attributes:
label: System information
description: |
* Nextflow version _(eg. 21.10.3)_
* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
* Version of nf-core/methylseq _(eg. 1.1, 1.5, 1.8.2)_
1 change: 0 additions & 1 deletion .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
blank_issues_enabled: false
contact_links:
- name: Join nf-core
url: https://nf-co.re/join
Expand Down
32 changes: 0 additions & 32 deletions .github/ISSUE_TEMPLATE/feature_request.md

This file was deleted.

11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
name: Feature request
description: Suggest an idea for the nf-core/methylseq pipeline
labels: enhancement
body:
- type: textarea
id: description
attributes:
label: Description of feature
description: Please describe your suggestion for a new feature. It might help to describe a problem or use case, plus any alternatives that you have considered.
validations:
required: true
10 changes: 4 additions & 6 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,15 @@ Remember that PRs should be made against the dev branch, unless you're preparing

Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/methylseq/tree/master/.github/CONTRIBUTING.md)
-->
<!-- markdownlint-disable ul-indent -->

## PR checklist

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - add to the software_versions process and a regex to `scrape_software_versions.py`
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](<https://github.com/>nf-core/methylseq/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/methylseq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint .`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker`).
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/methylseq/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/methylseq _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
Expand Down
12 changes: 0 additions & 12 deletions .github/markdownlint.yml

This file was deleted.

36 changes: 19 additions & 17 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -1,32 +1,20 @@
name: nf-core AWS full size tests
# This workflow is triggered on published releases.
# It can be additionally triggered manually with GitHub actions workflow dispatch.
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
# It runs the -profile 'test_full' on AWS batch

on:
workflow_run:
workflows: ["nf-core Docker push (release)"]
types: [completed]
release:
types: [published]
workflow_dispatch:


env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
TOWER_ACCESS_TOKEN: ${{ secrets.AWS_TOWER_TOKEN }}
AWS_JOB_DEFINITION: ${{ secrets.AWS_JOB_DEFINITION }}
AWS_JOB_QUEUE: ${{ secrets.AWS_JOB_QUEUE }}
AWS_S3_BUCKET: ${{ secrets.AWS_S3_BUCKET }}


jobs:
run-awstest:
run-tower:
name: Run AWS full tests
if: github.repository == 'nf-core/methylseq'
runs-on: ubuntu-latest
strategy:
matrix:
aligner: ['bismark', 'bismark_hisat', 'bwameth']
aligner: ["bismark", "bismark_hisat", "bwameth"]
steps:
- name: Setup Miniconda
uses: conda-incubator/setup-miniconda@v2
Expand All @@ -36,6 +24,20 @@ jobs:
- name: Install awscli
run: conda install -c conda-forge awscli
- name: Start AWS batch job
uses: nf-core/tower-action@v3
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/methylseq/work-${{ github.sha }}
parameters: |
{
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/methylseq/results-${{ github.sha }}"
}
profiles: test_full,aws_tower
nextflow_config: |
process.errorStrategy = 'retry'
process.maxRetries = 3
# Do a full-scale run with each of the three aligners
run: |
aws batch submit-job \
Expand Down
Loading