Move nf-test files and container definitions #15

drpatelh · 2023-11-30T13:24:44Z

Removed container.config and moved container definition back into process
Deleted empty nextflow.config files
Changed registry info where required
Updates CI to run nf-test
Remove params.publish_dir_mode from some processes because it seemed to break stuff?

adamrtalbot

Stuff for future discussions I wanted to write down while I was looking at it, but nothing wrong with the code itself.

adamrtalbot · 2023-11-30T13:39:37Z

nextflow.config

@@ -58,7 +58,7 @@ profiles {
    wave {
        wave.enabled = true
        wave.strategy = ['conda']


If you change this to ['conda', 'container', 'dockerfile', 'spack'] it will use these in that order, so a user could use Wave to copy the existing container without rebuilding.

Makes sense. You want to push that change?

nextflow.config

modules/local/seqera_runs_dump/main.nf

nextflow.config

adamrtalbot

Just checked and there are lots of errors here about missing files and invalid tests.

adamrtalbot · 2023-11-30T14:00:30Z

No .nf-tests triggered?

pinin4fjords · 2023-12-01T10:21:10Z

Why the move of the test file? This was a top-level 'pipeline' test (patterned after https://github.com/nf-core/fetchngs/blob/master/tests/main.nf.test etc). The move changes which main.nf is referenced, so there may be other changes to make, no?

workflows/nf_aggregate/tests/main.nf.test

adamrtalbot · 2023-12-01T17:01:01Z

Moved pipeline level test back re: #14 (comment)

adamrtalbot

It's perfect because I wrote most of it.

drpatelh · 2023-12-01T23:13:53Z

Ok. Tidied things up a little. Things left to do that be good to sort out before the release.

Update the MultiQC nf-test on nf-core modules to remove the dependency for FASTQC so we don't have to install the module. Doesn't make sense installing it here
Add workflow test to workflows/nf_aggregate/tests

What else was left @pinin4fjords ?

pinin4fjords · 2023-12-04T09:40:16Z

What else was left @pinin4fjords ?

Subworkflow test? I'll get on the workflow one anyway

pinin4fjords · 2023-12-05T09:47:18Z

MultiQC module groundwork:

Add fastqc outputs for testing multiqc nf-core/test-datasets#1054 for test data addition
MultiQC module tests update: Pull multiqc static multiqc test data nf-core/modules#4539

This reverts commit 2475194.

adamrtalbot · 2023-12-05T12:08:01Z

We seem to be skipping tests fairly frequently leading to false positive results. I've added it as an issue: #16

For now, be careful when accepting a PR and make sure it's ran all tests.

pinin4fjords · 2023-12-05T12:10:54Z

We seem to be skipping tests fairly frequently leading to false positive results. I've added it as an issue: #16

For now, be careful when accepting a PR and make sure it's ran all tests.

Ahh yes, sorry, this was me, and you're right, the strategy makes no sense here.

pinin4fjords

Don't merge yet, mucking about with the tags

.github/workflows/ci.yml

modules/local/plot_run_gantt/tests/main.nf.test

.github/workflows/ci.yml

nf-test.config

tests/nextflow.config

Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com>

…le debug

drpatelh · 2023-12-06T17:04:00Z

Move nf-test files and container definitions

4566b3d

adamrtalbot approved these changes Nov 30, 2023

View reviewed changes

adamrtalbot requested changes Nov 30, 2023

View reviewed changes

adamrtalbot added 5 commits November 30, 2023 14:03

Fix CI test triggers (which weren't active on PR)

d27f656

Fix mistake in CI

0d8c84c

Add FASTQC for testing purposes

a0af0a4

Load test data

e50384e

Trying to load an extra param into tests

fad7a36

pinin4fjords reviewed Dec 1, 2023

View reviewed changes

workflows/nf_aggregate/tests/main.nf.test Outdated Show resolved Hide resolved

adamrtalbot added 2 commits December 1, 2023 16:50

Move pipeline level test back to pipeline folder

4fdb050

Remove publish_dir_mode use because it broke stuff

6b0f457

adamrtalbot approved these changes Dec 1, 2023

View reviewed changes

drpatelh added 6 commits December 1, 2023 13:24

Remove FastQC module

cb63b7a

Add wave.strategy precedence

8d271f8

Remove Conda profile from testing in CI

cf8b31e

Add publish_dir_mode parameter back

e7216d0

Remove lib path

36b9cd8

Remove config for test data

2475194

pinin4fjords added 5 commits December 4, 2023 15:25

Add workflow tests

031c97a

Contains check for variable HTML

0dbc431

Switch to contains

58ed655

Add subworkflow tests

ad11443

Remove unused params

000be55

Bump to multiqc without fastqc

255a6d1

adamrtalbot self-requested a review December 5, 2023 11:56

pinin4fjords added 3 commits December 5, 2023 11:57

Revert "Remove config for test data"

155c629

This reverts commit 2475194.

Bump modules sha for test data

23c0a05

Fix typo

13e6b4b

adamrtalbot mentioned this pull request Dec 5, 2023

Testing strategy incorrectly skips tests #16

Closed

pinin4fjords requested changes Dec 5, 2023

View reviewed changes

adamrtalbot reviewed Dec 5, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

modules/local/plot_run_gantt/tests/main.nf.test Outdated Show resolved Hide resolved

.github/workflows/ci.yml Show resolved Hide resolved

pinin4fjords force-pushed the release_fixes branch from db3a957 to 13e6b4b Compare December 5, 2023 15:53

pinin4fjords added 5 commits December 5, 2023 18:51

Try again to bypass the filtering

7daa4e8

Fix env -> out

7f87368

Fix output bit

651eda4

fix up yq actions usage

19f42db

Tweaks

ec14aa0

pinin4fjords approved these changes Dec 6, 2023

View reviewed changes

adamrtalbot and others added 5 commits December 6, 2023 10:52

Remove reports from nf-test configuration

dec50d5

Remove params from nf-aggregate workflow

68ccda2

Remove explicit params.publish_dir_mode definition in tests

d5cd848

Update UTILS_NFVALIDATION_PLUGIN subworkflow

d9521fa

Add final CI test to confirm all nf-tests passed

64fb120

maxulysse reviewed Dec 6, 2023

View reviewed changes

nf-test.config Outdated Show resolved Hide resolved

maxulysse reviewed Dec 6, 2023

View reviewed changes

tests/nextflow.config Show resolved Hide resolved

drpatelh commented Dec 6, 2023

View reviewed changes

tests/nextflow.config Show resolved Hide resolved

pinin4fjords and others added 3 commits December 6, 2023 15:55

Try workdir thing

20d8a3a

Co-authored-by: Adam Talbot <12817534+adamrtalbot@users.noreply.github.com>

try tmpdir thing without closure

520fd12

Ignore process selector warnings by default in favour of using -profi…

9af0c7e

…le debug

drpatelh merged commit 817ea2f into main Dec 6, 2023
22 checks passed

adamrtalbot deleted the release_fixes branch December 6, 2023 17:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move nf-test files and container definitions #15

Move nf-test files and container definitions #15

drpatelh commented Nov 30, 2023 •

edited by adamrtalbot

Loading

adamrtalbot left a comment

adamrtalbot Nov 30, 2023

drpatelh Nov 30, 2023

adamrtalbot left a comment

adamrtalbot commented Nov 30, 2023

pinin4fjords commented Dec 1, 2023 •

edited

Loading

adamrtalbot commented Dec 1, 2023

adamrtalbot left a comment

drpatelh commented Dec 1, 2023 •

edited by adamrtalbot

Loading

pinin4fjords commented Dec 4, 2023

pinin4fjords commented Dec 5, 2023 •

edited

Loading

adamrtalbot commented Dec 5, 2023 •

edited

Loading

pinin4fjords commented Dec 5, 2023

pinin4fjords left a comment

drpatelh commented Dec 6, 2023

Move nf-test files and container definitions #15

Move nf-test files and container definitions #15

Conversation

drpatelh commented Nov 30, 2023 • edited by adamrtalbot Loading

adamrtalbot left a comment

Choose a reason for hiding this comment

adamrtalbot Nov 30, 2023

Choose a reason for hiding this comment

drpatelh Nov 30, 2023

Choose a reason for hiding this comment

adamrtalbot left a comment

Choose a reason for hiding this comment

adamrtalbot commented Nov 30, 2023

pinin4fjords commented Dec 1, 2023 • edited Loading

adamrtalbot commented Dec 1, 2023

adamrtalbot left a comment

Choose a reason for hiding this comment

drpatelh commented Dec 1, 2023 • edited by adamrtalbot Loading

pinin4fjords commented Dec 4, 2023

pinin4fjords commented Dec 5, 2023 • edited Loading

adamrtalbot commented Dec 5, 2023 • edited Loading

pinin4fjords commented Dec 5, 2023

pinin4fjords left a comment

Choose a reason for hiding this comment

drpatelh commented Dec 6, 2023

drpatelh commented Nov 30, 2023 •

edited by adamrtalbot

Loading

pinin4fjords commented Dec 1, 2023 •

edited

Loading

drpatelh commented Dec 1, 2023 •

edited by adamrtalbot

Loading

pinin4fjords commented Dec 5, 2023 •

edited

Loading

adamrtalbot commented Dec 5, 2023 •

edited

Loading