Releases: nf-core/tools
v2.8 - Ruthenium Monkey
v2.8 - Ruthenium Monkey - [2023-04-27]
Template
- Explicitly disable
conda
when a container profile (#2140) - Turn on automatic clean up of intermediate files in
work/
on successful pipeline completion in full-test config (#2163) [Contributed by @jfy133] - Add documentation to
usage.md
on how to useparams.yml
files, based on nf-core/ampliseq text (#2173) [Contributed by @jfy133, @d4straub] - Make jobs automatically resubmit for a much wider range of exit codes (now
104
and130..145
) (#2170) - Add a clean-up GHA which closes issues and PRs with specific labels (#2183)
- Remove problematic sniffer code in samplesheet_check.py that could give false positive 'missing header' errors ([https://github.com//pull/2194]) [Contributed by @Midnighter, @jfy133]
- Consistent syntax for branch checks in PRs (#2202)
- Fixed minor Jinja2 templating bug that caused the PR template to miss a newline
- Updated AWS tests to use newly moved
seqeralabs/action-tower-launch
instead ofnf-core/tower-action
- Remove
.cff
files from.editorconfig
[(#2145)[https://github.com//pull/2145]] - Simplify pipeline README (#2186)
- Added support for the apptainer container engine via
-profile apptainer
. (#2244) [Contributed by @jfy133] - Added config
docker.registry
to pipeline template for a configurable default container registry when using Docker containers. Defaults toquay.io
(#2133) - Add tower.yml file to the pipeline template (#2251)
- Add mastodon badge to README (#2253)
- Removed
quay.io
from all module Docker container references as this is now supplied at pipeline level. (#2249) - Remove
CITATION.cff
file from pipeline template, to avoid that pipeline Zenodo entries reference the nf-core publication instead of the pipeline (#2059).
Linting
- Update modules lint test to fail if enable_conda is found (#2213)
- Read module lint configuration from
.nf-core.yml
, not.nf-core-lint.yml
(#2221) nf-core schema lint
now defaults to lintingnextflow_schema.json
if no filename is provided (#2225)- Warn if
/zenodo.XXXXXX
is present in the Readme (#2254) - Lint all labels in a module (#2227)
Modules
- Add an
--empty-template
option to create a module without TODO statements or examples (#2175 & #2177) - Removed the
nf-core modules mulled
command and all its code dependencies (2199). - Take into accout the provided
--git_remote
URL when linting all modules (2243).
Subworkflows
- Fixing problem when a module included in a subworkflow had a name change from TOOL to TOOL/SUBTOOL (#2177)
- Fix
nf-core subworkflows test
not running subworkflow tests (#2181) - Add tests for
nf-core subworkflows create-test-yml
(#2219)
General
- Deprecate Python 3.7 support because it reaches EOL (#2210)
nf-core modules/subworkflows info
now prints the include statement for the module/subworkflow (#2182).- Add a clean-up GHA which closes issues and PRs with specific labels (#2183)
- update minimum version of rich to 13.3.1 (#2185)
- Add the Nextflow version to Gitpod container matching the minimal Nextflow version for nf-core (according to
nextflow.config
) (#2196) - Use
nfcore/gitpod:dev
container in the dev branch (#2196) - Replace requests_mock with responses in test mocks (#2165).
- Add warning when installing a module from an
org_path
that exists in multiple remotes inmodules.json
(#2228 #2239). - Add the possibility to translate refgenie asset aliases to the ones used in a pipeline with an alias_translations.yaml file (#2242).
- Add initial CHM13 support (1988)
v2.7.2 - Mercury Eagle Patch
v2.7.2 - Mercury Eagle Patch
Patch release solving some bugs introduced with v2.7
Template
- Fix the syntax of github_output in GitHub actions (#2114)
- Fix a bug introduced in 2.7 that made pipelines hang (#2132)
Linting
- Allow specifying containers in less than three lines (#2121)
- Run prettier after dumping a JSON schema file (#2124)
General
v2.7.1 - Mercury Eagle Patch
This release fixes a sync error with the new syntax of set_output
on GitHub Actions.
- Patch release to fix pipeline sync (#2110)
v2.7 - Mercury Eagle
Another big release with lots of new features and bug fixes. Thanks to all contributors!
🌟 Highlights
- New
nf-core subworkflows
subcommand for creating, removing, testing, updating and finding subworkflows, see the documentation for more information. - Every pipeline has now it's own GitHub codespace template, which can be used to develop the pipeline directly in the browser.
- Improved handling of modules and subworkflows from other repos than nf-core/modules.
- Pre-commit is now installed as a dependency, which allows us, besides other things, to run prettier on the fly even if it is not manually installed.
- Shell completion for nf-core commands, more information here.
Template
Features
- Ignore files in
bin/
directory when running prettier (#2080). - Add GitHub codespaces template (#1957)
nextflow run <pipeline> --version
will now print the workflow version from the manifest and exit (#1951).- Add profile for running
docker
with the ARM chips (including Apple silicon) (#1942 and #2034). - Flip execution order of parameter summary printing and parameter validation to prevent 'hiding' of parameter errors (#2033).
- Change colour of 'pipeline completed successfully, but some processes failed' from red to yellow (#2096).
Bug fixes
- Fix lint warnings for
samplesheet_check.nf
module (#1875). - Check that the workflow name provided with a template doesn't contain dashes (#1822)
- Remove
CITATION.cff
file from pipeline template, to avoid that pipeline Zenodo entries reference the nf-core publication instead of the pipeline (#2059).
Linting
Features
- Add
--sort-by
option to linting which allows ordering module lint warnings/errors by either test name or module name (#2077).
Bug fixes
- Don't lint pipeline name if
manifest.name
in.nf-core.yml
(#2035) - Don't check for
docker pull
commands inactions_ci
lint test (leftover from DSL1) (#2055).
General
Features
- Use pre-commit run prettier if prettier is not available (#1983) and initialize pre-commit in gitpod and codespaces (#1957).
- Refactor CLI flag
--hide-progress
to be at the top-level group, like--verbose
(#2016) nf-core sync
now supports the template YAML file using-t/--template-yaml
(#1880).- The default branch can now be specified when creating a new pipeline repo #1959.
- Only warn when checking that the pipeline directory contains a
main.nf
and anextflow.config
file if the pipeline is not an nf-core pipeline #1964 - Bump promoted Python version from 3.7 to 3.8 (#1971).
- Extended the chat notifications to Slack (#1829).
- Don't print source file + line number on logging messages (except when verbose) (#2015)
- Automatically format
test.yml
content with Prettier (#2078) - Automatically format
modules.json
content with Prettier (#2074) - Add shell completion for nf-core tools commands(#2070)
Bug fixes, maintenance and tests
- Fix error in tagging GitPod docker images during releases (#1874).
- Fix bug when updating modules from old version in old folder structure (#1908).
- Don't remove local copy of modules repo, only update it with fetch (#1881).
- Improve test coverage of
sync.py
and__main__.py
(#1936, #1965). - Add file
versions.yml
when generatingtest.yml
withnf-core modules create-test-yml
but don't check for md5sum #1963. - Mock biocontainers and anaconda api calls in modules and subworkflows tests #1967
- Run tests with Python 3.11 (#1970).
- Run test with a realistic version of git (#2043).
- Fix incorrect file deletion in
nf-core launch
when--params_in
has the same name as--params_out
(#1986). - Updated GitHub actions (#1998, #2001)
- Code maintenance (#1818, #2032, #2073).
- Track from where modules and subworkflows are installed (#1999).
- Substitute ModulesCommand and SubworkflowsCommand by ComponentsCommand (#2000).
- Prevent installation with unsupported Python versions (#2075).
- Allow other remote URLs not starting with
http
(#2061)
Modules
- Update patch file paths if the modules directory has the old structure (#1878).
- Don't write to
modules.json
file when applying a patch file duringnf-core modules update
(#2017).
Subworkflows
- Add subworkflow commands
create-test-yml
,create
andinstall
(#1897). - Update subworkflows install so it installs also imported modules and subworkflows (#1904).
check_up_to_date()
function frommodules_json.py
also checks for subworkflows (#1934).- Add tests for
nf-core subworkflows install
command (#1996). - Function
create()
frommodules_json.py
adds also subworkflows tomodules.json
file (#2005). - Add
nf-core subworkflows update
command (#2019).
v2.6 - Tin Octopus
⚠️ Please read
This release goes together with a major update on nf-core/modules that restructures DSL2 module files and updates the syntax of modules.json
.
These changes have been brought about to accommodate the imminent release of shared subworkflows.
Please be aware of some points:
- ♻️ You will need to run
nf-core modules update
on your pipeline which should move a lot of files around - 💀 Any module
.patch
files fromnf-core modules patch
may be lost (if you have any). Please take care to regenerate them.
If you have any issues, please drop us a line on Slack on the #tools
channel 👍🏻
Template
- Add
actions/upload-artifact
step to the awstest workflows, to expose the debug log file - Add
prettier
as a requirement to Gitpod Dockerimage - Bioconda incompatible conda channel setups now result in more informative error messages (#1812)
- Update MultiQC module, update supplying MultiQC default and custom config and logo files to module
- Add a 'recommend' methods description text to MultiQC to help pipeline users report pipeline usage in publications (#1749)
- Fix template spacing modified by JINJA (#1830)
- Fix MultiQC execution on template #1855
- Don't skip including
base.config
when skipping nf-core/configs
Linting
- Pipelines: Check that the old renamed
lib
files are not still present:Checks.groovy
->Utils.groovy
Completion.groovy
->NfcoreTemplate.groovy
Workflow.groovy
->WorkflowMain.groovy
General
- Add function to enable chat notifications on MS Teams, accompanied by
hook_url
param to enable it. - Schema: Remove
allOf
if no definition groups are left. - Use contextlib to temporarily change working directories (#1819)
- More helpful error messages if
nf-core download
can't parse a singularity image download
Modules
- If something is wrong with the local repo cache, offer to delete it and try again (#1850)
- Restructure code to work with the directory restructuring in modules (#1859)
- Make
label: process_single
default when creating a new module
Full Changelog
v2.5.1 - Gold Otter Patch
v2.5 - Gold Otter
Full changelog: 2.4.1...2.5
Template
- Bumped Python version to 3.7 in the GitHub linting in the workflow template (#1680)
- Fix bug in pipeline readme logo URL (#1590)
- Switch CI to use setup-nextflow action to install Nextflow (#1650)
- Add
CITATION.cff
#361 - Add Gitpod and Mamba profiles to the pipeline template (#1673)
- Remove call to
getGenomeAttribute
inmain.nf
when runningnf-core create
without iGenomes (#1670) - Make
nf-core create
fail if Git default branch name is dev or TEMPLATE (#1705) - Convert
console
snippets tobash
snippets in the template where applicable (#1729) - Add
branch
field to module entries inmodules.json
to record what branch a module was installed from (#1728) - Add customisation option to remove all GitHub support with
nf-core create
(#1766)
Linting
- Check that the
.prettierignore
file exists and that starts with the same content. - Update
readme.py
nf version badge validation regexp to accept any signs before version number (#1613) - Add isort configuration and GitHub workflow (#1538)
- Use black also to format python files in workflows (#1563)
- Add check for mimetype in the
input
parameter. (#1647) - Check that the singularity and docker tags are parsable. Add
--fail-warned
flag tonf-core modules lint
(#1654) - Handle exception in
nf-core modules lint
when process name doesn't start with process (#1733)
General
- Remove support for Python 3.6 (#1680)
- Add support for Python 3.9 and 3.10 (#1680)
- Invoking Python with optimizations no longer affects the program control flow (#1685)
- Update
readme
to drop--key
option fromnf-core modules list
and add the new pattern syntax - Add
--fail-warned
flag tonf-core lint
to make warnings fail (#1593) - Add
--fail-warned
flag to pipeline linting workflow (#1593) - Updated the nf-core package requirements (#1620, #1757, #1756)
- Remove dependency of the mock package and use unittest.mock instead (#1696)
- Fix and improve broken test for Singularity container download (#1622)
- Use
$XDG_CACHE_HOME
or~/.cache
instead of$XDG_CONFIG_HOME
or~/config/
as base directory for API cache - Switch CI to use setup-nextflow action to install Nextflow (#1650)
- Add tests for
nf-core modules update
andModulesJson
. - Add CI for GitLab remote #1646
- Add
CITATION.cff
#361 - Allow customization of the
nf-core
pipeline template when usingnf-core create
(#1548) - Add Refgenie integration: updating of nextflow config files with a refgenie database (#1090)
- Fix
--key
option innf-core lint
when supplying a module lint test name (#1681) - Add
no_git=True
when creating a new pipeline and initialising a git repository is not needed innf-core lint
andnf-core bump-version
(#1709) - Move
strip_ansi_code
function in lint toutils.py
- Simplify control flow and don't use equality comparison for
None
and booleans - Replace use of the deprecated
distutils
Version object with that frompackaging
(#1735) - Add code to cancel CI run if a new run starts (#1760)
- CI for the API docs generation now uses the ubuntu-latest base image (#1762)
- Add option to hide progress bars in
nf-core lint
andnf-core modules lint
with--hide-progress
.
Modules
- Add
--fix-version
flag tonf-core modules lint
command to update modules to the latest version (#1588) - Fix a bug in the regex extracting the version from biocontainers URLs (#1598)
- Update how we interface with git remotes. (#1626)
- Add prompt for module name to
nf-core modules info
(#1644) - Update docs with example of custom git remote (#1645)
- Command
nf-core modules test
obtains module name suggestions from installed modules (#1624) - Add
--base-path
flag tonf-core modules
to specify the base path for the modules in a remote. Also refactoredmodules.json
code. (#1643) Removed after (#1754) - Rename methods in
ModulesJson
to remove explicit reference tomodules.json
- Fix inconsistencies in the
--save-diff
flagnf-core modules update
. Refactornf-core modules update
(#1536) - Fix bug in
ModulesJson.check_up_to_date
causing it to ask for the remote of local modules - Handle errors when updating module version with
nf-core modules update --fix-version
(#1671) - Make
nf-core modules update --save-diff
work when files were created or removed (#1694) - Get the latest common build for Docker and Singularity containers of a module (#1702)
- Add short option for
--no-pull
option innf-core modules
- Add
nf-core modules patch
command (#1312) - Add support for patch in
nf-core modules update
command (#1312) - Add support for patch in
nf-core modules lint
command (#1312) - Add support for custom remotes in
nf-core modules lint
(#1715) - Make
nf-core modules
commands work with arbitrary git remotes (#1721) - Add links in
README.md
forinfo
andpatch
commands (#1722]) - Fix misc. issues with
--branch
and--base-path
(#1726) - Add
branch
field to module entries inmodules.json
to record what branch a module was installed from (#1728) - Fix broken link in
nf-core modules info
(#1745) - Fix unbound variable issues and minor refactoring #1742
- Recreate modules.json file instead of complaining about incorrectly formatted file. (#1741
- Add support for patch when creating
modules.json
file (#1752)
v2.4.1 - Cobolt Koala Patch
Somehow it wouldn't feel like a real nf-core/tools release if there wasn't some error with the automatic template sync that required an immediate patch release.. 🤔
Hopefully this release will fill everyone's inboxes with automated update pull requests.. 🤞🏻
v2.4 - Cobolt Koala
This release started off as a patch release to fix some errors introduced in 2.3 (eg. pipeline email template errors). However, it slowly collected enough new features to warrant a proper minor release of its own.
Many thanks to everyone who contributed!
Full Changelog: 2.3.2...2.4
Template
- ✨ Add actions workflow to respond to
@nf-core-bot fix linting
comments on pipeline PRs - Fix Prettier formatting bug in completion email HTML template (#1509)
- Fix bug in pipeline readme logo URL
- Set the default DAG graphic output to HTML to have a default that does not depend on Graphviz being installed on the host system (#1512).
- Removed retry strategy for AWS tests CI, as Nextflow now handles spot instance retries itself
- Add
.prettierignore
file to stop Prettier linting tests from running over test files - Made module template test command match the default used in
nf-core modules create-test-yml
(#1562) - Removed black background from Readme badges now that GitHub has a dark mode, added Tower launch badge.
- Don't save md5sum for
versions.yml
when runningnf-core modules create-test-yml
(#1511) - Read entire lines when sniffing the samplesheet format (fix #1561)
General
- ✨ Add actions workflow to respond to
@nf-core-bot fix linting
comments on nf-core/tools PRs - Use
$XDG_CONFIG_HOME
or~/.config/nf-core
instead of~/.nfcore
for API cache (the latter can be safely deleted) - Consolidate GitHub API calls into a shared function that uses authentication from the
gh
GitHub cli tool orGITHUB_AUTH_TOKEN
to avoid rate limiting (#1499) - Add an empty line to
modules.json
,params.json
andnextflow-schema.json
when dumping them to avoid prettier errors. - Remove empty JSON schema definition groups to avoid usage errors (#1419)
- Bumped the minimum version of
rich
fromv10
tov10.7.0
Modules
- ✨ Add a new command
nf-core modules mulled
which can generate the name for a multi-tool container image. - ✨ Add a new command
nf-core modules test
which runs pytests locally. - ✨ Print include statement to terminal when
modules install
(#1520) - Allow follow links when generating
test.yml
file withnf-core modules create-test-yml
(1570) - Escaped test run output before logging it, to avoid a rich
MarkupError
Linting
- Don't allow a
.nf-core.yaml
file, should be.yml
(#1515). shell
blocks now recognised to avoid errorwhen: condition has too many lines
(#1557)- Fixed error when using comments after
input
tuple lines (#1542) - Don't lint the
shell
block whenscript
is used (1558) - Check that
template
is used inscript
blocks - Tweaks to CLI output display of lint results
v2.3.2 - Mercury Vulture Fixed Formatting
Very minor patch release to fix the full size AWS tests and re-run the template sync, which partially failed due to GitHub pull-requests being down at the time of release.
Template
- Updated the AWS GitHub actions to let nf-core/tower-action use it's defaults for pipeline and git sha (#1488)
- Add prettier editor extension to
gitpod.yml
in template (#1485) - Remove traces of markdownlint in the template (#1486
- Remove accidentally added line in
CHANGELOG.md
in the template (#1487) - Update linting to check that
.editorconfig
is there and.yamllint.yml
isn't.