Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure pipeline tests and move pipeline linting into subfolder #3070

Merged
merged 92 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from 51 commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
e238691
move pipelines tests into `pipelines` subdirectory to mirror command …
mashehu Jul 15, 2024
891d08a
move lint tests into pipeline folder
mashehu Jul 16, 2024
11130e6
make pipeline tests a subclass of TestPipeline
mashehu Jul 16, 2024
20f0ba6
move pytest.ini into pyproject.toml
mashehu Jul 16, 2024
36050fd
migrate more tests to pathilb, use commonly shared TEST_DATA_DIR var
mashehu Jul 16, 2024
fb27410
use testpipeline class in test_lint
mashehu Jul 16, 2024
1d2076b
fix tests
mashehu Jul 16, 2024
65084dd
[automated] Update CHANGELOG.md
nf-core-bot Jul 16, 2024
2a9205e
Merge branch 'dev' of github.com:nf-core/tools into move-pipeline-lin…
mashehu Jul 16, 2024
0491cf5
Merge branch 'move-pipeline-linting' of github.com:mashehu/tools into…
mashehu Jul 16, 2024
45fdfb9
more fine-grained setup steps
mashehu Jul 17, 2024
189015f
don't run list_files on every init, only needed for one linting step
mashehu Jul 17, 2024
2ddc860
create git repo with testpipeline
mashehu Jul 17, 2024
c313fed
update textual snapshots to new location
mashehu Jul 17, 2024
d656065
fix tests
mashehu Jul 17, 2024
21cdbbd
remove asyncio mode (and see what happens)
mashehu Jul 17, 2024
e0a82d5
fix refgenie tests
mashehu Jul 17, 2024
1fde6ca
add cleanup step to some download tests
mashehu Jul 17, 2024
a0f788d
start converting linting tests to new subclass structure
mashehu Jul 17, 2024
13be56f
update prettier version
mashehu Jul 17, 2024
da155ab
import linting tests correctly to avoid "module is not callable" warn…
mashehu Jul 17, 2024
4c77621
convert rest of the lint test to new subclass structure
mashehu Jul 17, 2024
fcc43fd
add more tests to CI
mashehu Jul 17, 2024
a779d12
simplify sed command
mashehu Jul 17, 2024
ca0769b
fix ci tests
mashehu Jul 17, 2024
768cfc3
find ALL test files in CI
mashehu Jul 17, 2024
3aee9cb
add pytest-asyncio
mashehu Jul 17, 2024
09b0623
convert subworkflow tests to new structure
mashehu Jul 17, 2024
f7be79d
migrate modules tests to new structure
mashehu Jul 18, 2024
d95f7f8
avoid naming collision with variable name `dir`
mashehu Jul 18, 2024
60973a1
fix ALL the mypy errors
mashehu Jul 19, 2024
61898dc
fix rich imports, remove boolean type for `installed_by`, fix more oc…
mashehu Jul 22, 2024
47229ff
fix types and tests
mashehu Jul 22, 2024
ff1a495
avoid circular import due to modules_json being importetd
mashehu Jul 24, 2024
0cf5f8b
remove broken _repr_ (breaks because not all parameters are intilaized)
mashehu Jul 25, 2024
879e808
add types
mashehu Jul 25, 2024
178146f
fix export of pydantic models
mashehu Jul 25, 2024
d0a968c
fix tests
mashehu Jul 25, 2024
33f461a
migrate to pathlib
mashehu Jul 25, 2024
1473611
change import strategy for lint tests
mashehu Jul 25, 2024
9dc61d6
migrate to pathlib
mashehu Jul 25, 2024
0a3cb5f
fix mypy linting
mashehu Jul 25, 2024
b7b45df
fix circular import
mashehu Jul 25, 2024
c6227a7
fix module command imports
mashehu Jul 25, 2024
847c683
fix: cannot import name 'NotRequired'
mashehu Jul 25, 2024
e7ac781
more dir-> directory conversions
mashehu Jul 25, 2024
5a8ae12
fix pydantic warnings
mashehu Jul 25, 2024
ac1ea23
fix notrequired not found
mashehu Jul 25, 2024
79727c0
fix pydantic type
mashehu Jul 25, 2024
ee67c5f
fix before model creation
mashehu Jul 25, 2024
dae348e
fix modules repo import error
mashehu Jul 25, 2024
b53a4b8
fix incorrect type name
mashehu Jul 26, 2024
bdbdd17
fix types
mashehu Jul 26, 2024
98c3fb0
fix type definition for older python version
mashehu Jul 26, 2024
7b883eb
fix incorrect types and missing cli option names
mashehu Jul 26, 2024
0fe3910
fix create-logo cli command
mashehu Jul 26, 2024
ce32536
add types, include review comments, add pydantic mypy plugin
mashehu Jul 26, 2024
8eeaf28
fix mypy error
mashehu Jul 26, 2024
95de96b
allow dashes in pipeilne short name (why didn't this fail before?)
mashehu Jul 26, 2024
1b6f231
Revert "allow dashes in pipeilne short name (why didn't this fail bef…
mashehu Jul 26, 2024
5186ac7
use path.walk correctly
mashehu Jul 26, 2024
8733987
Revert "use path.walk correctly"
mashehu Jul 26, 2024
1679594
switch back to os.walk()
mashehu Jul 26, 2024
3c0433e
fix sync
mashehu Jul 26, 2024
4d6930b
fix types
mashehu Jul 26, 2024
26b1cd1
set force true in sync to create a template file
mashehu Jul 29, 2024
59a3eb3
fix missing initialization in download.py
mashehu Jul 29, 2024
fec536f
Apply suggestions from code review
mashehu Jul 29, 2024
7f03f9e
fix mypy warnings
mashehu Jul 29, 2024
d2020f5
set default value for self.directory and more dir->directory switches
mashehu Jul 29, 2024
ebdb398
remove unnecessary checks
mashehu Jul 29, 2024
15a9071
remove unnecessary Path conversions
mashehu Jul 29, 2024
fa00b1b
fix types for 3.8
mashehu Jul 29, 2024
e7f06be
limit `component` to type string
mashehu Jul 29, 2024
ae7d912
handle more type warnings
mashehu Jul 29, 2024
d7587dc
fix import errors, handle outdir as string
mashehu Jul 29, 2024
5979b9d
add error message and resolve circular import
mashehu Jul 29, 2024
24a3ece
actually test installation with a correct subworkflow
mashehu Jul 29, 2024
05def59
fix list and info command
mashehu Jul 29, 2024
8de3871
exclude click from rich traceback
mashehu Jul 30, 2024
8fe4be6
give nicer message on install failure, fix install test
mashehu Jul 30, 2024
6bc7361
use self.registry
mashehu Jul 30, 2024
e4717cc
move subworkflows main_nf linting closer to the modules version
mashehu Jul 30, 2024
b022862
fix handling of missing dir in list command
mashehu Jul 30, 2024
e9bf654
remove unnecessary string conversion
mashehu Jul 30, 2024
d4c06cb
fix tests
mashehu Jul 30, 2024
d57c815
add type hints to utils functions
mashehu Jul 30, 2024
495e05a
fix refgenie tests
mashehu Jul 30, 2024
4c07bdf
fix tests
mashehu Jul 30, 2024
4f75538
update ruff
mashehu Aug 5, 2024
b83da1f
Merge branch 'dev' of github.com:nf-core/tools into move-pipeline-lin…
mashehu Aug 12, 2024
d4d2314
reset lintconfigtype
mashehu Aug 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ indent_style = unset
[**/Makefile]
indent_style = unset

[tests/__snapshots__/*]
[tests/pipelines/__snapshots__/*]
charset = unset
end_of_line = unset
insert_final_newline = unset
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ jobs:
- name: List tests
id: list_tests
run: |
echo "tests=$(find tests/test_* | tac | sed 's/tests\///g' | jq -R -s -c '{test: (split("\n")[:-1])}')" >> $GITHUB_OUTPUT
echo "tests=$(find tests -type f -name "test_*.py" | tac | sed 's/tests\///g' | jq -R -s -c '{test: (split("\n")[:-1])}')" >> $GITHUB_OUTPUT
outputs:
tests: ${{ steps.list_tests.outputs.tests }}

Expand Down Expand Up @@ -149,10 +149,15 @@ jobs:
name: Snapshot Report ${{ matrix.test }}
path: ./snapshot_report.html

- name: remove slashes from test name
run: |
test=$(echo ${{ matrix.test }} | sed 's/\//__/g')
echo "test=${test}" >> $GITHUB_ENV

- name: Upload coverage
uses: actions/upload-artifact@65462800fd760344b1a7b4382951275a0abb4808 # v4
with:
name: coverage_${{ matrix.test }}
name: coverage_${{ env.test }}
path: .coverage

coverage:
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ repos:
hooks:
- id: prettier
additional_dependencies:
- prettier@3.2.5
- prettier@3.3.3

- repo: https://github.com/editorconfig-checker/editorconfig-checker.python
rev: "2.7.3"
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

- Fix linting fail on nfcore_external_java_deps if nf_schema is used ([#2976](https://github.com/nf-core/tools/pull/2976))
- Conda module linting: Include package name in log file ([#3014](https://github.com/nf-core/tools/pull/3014))
- Rrestructure pipeline tests and move pipeline linting into subfolder ([#3070](https://github.com/nf-core/tools/pull/3070))
mashehu marked this conversation as resolved.
Show resolved Hide resolved

### Download

Expand Down
Loading
Loading