Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update docs with latest changes #42

Merged
merged 13 commits into from
Aug 20, 2024
31 changes: 18 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@
> [!IMPORTANT]
> nf-schema is the new version of the now deprecated [nf-validation](https://github.com/nextflow-io/nf-validation). Please follow the [migration guide](https://nextflow-io.github.io/nf-schema/latest/migration_guide/) to migrate your code to this new version.

This [Nextflow plugin](https://www.nextflow.io/docs/latest/plugins.html#plugins) provides a number of functions that can be included into a Nextflow pipeline script to work with parameter and sample sheet schema. Using these functions you can:
This [Nextflow plugin](https://www.nextflow.io/docs/latest/plugins.html#plugins) provides functionality that can be used in a Nextflow pipeline to work with parameter and sample sheet schema. The added functionality is:

- 📖 Print usage instructions to the terminal (for use with `--help`)
- ✍️ Print log output showing parameters with non-default values
- ✅ Validate supplied parameters against the pipeline schema
- 📋 Validate the contents of supplied sample sheet files
- 🛠️ Create a Nextflow channel with a parsed sample sheet
- 🛠️ Create a type-casted list from a parsed sample sheet

Supported sample sheet formats are CSV, TSV, JSON and YAML.

Expand All @@ -25,27 +25,21 @@ Declare the plugin in your Nextflow pipeline configuration file:

```groovy title="nextflow.config"
plugins {
id 'nf-schema@2.0.0'
id 'nf-schema@2.1.0'
}
```

This is all that is needed - Nextflow will automatically fetch the plugin code at run time.

> [!NOTE]
> The snippet above will always try to install the latest version, good to make sure
> that the latest bug fixes are included! However, this can cause difficulties if running
> offline. You can pin a specific release using the syntax `nf-schema@2.0.0`
> The snippet above will always try to install the specified version. We encourage always pinning the
> plugin version to make sure the used pipeline will keep working when a new version of `nf-schema`
> with breaking changes has been released.

You can now include the plugin helper functions into your Nextflow pipeline:

```groovy title="main.nf"
include { validateParameters; paramsHelp; paramsSummaryLog; samplesheetToList } from 'plugin/nf-schema'

// Print help message, supply typical command line usage for the pipeline
if (params.help) {
log.info paramsHelp("nextflow run my_pipeline --input input_file.csv")
exit 0
}
include { validateParameters; paramsSummaryLog; samplesheetToList } from 'plugin/nf-schema'

// Validate input parameters
validateParameters()
Expand All @@ -57,10 +51,21 @@ log.info paramsSummaryLog(workflow)
ch_input = Channel.fromList(samplesheetToList(params.input, "assets/schema_input.json"))
```

Or enable the creation of the help message (using `--help`) in the configuration file:

```groovy title="nextflow.config"
validation {
help {
enabled: true
}
}
```

## Dependencies

- Java 11 or later
- <https://github.com/harrel56/json-schema>
- Nextflow 23.10.0 or later

## Slack channel

Expand Down
2 changes: 1 addition & 1 deletion docs/background.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Background
description: Get up and running with nf-schema
description: Background information about nf-schema
hide:
- toc
---
Expand Down
106 changes: 105 additions & 1 deletion docs/configuration/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ validation {
}
```

## parametersSchema

This option can be used to set the parameters JSON schema to be used by the plugin. This will affect parameter validation (`validateParameters()`), the summary logs (`paramsSummaryLog()` and `paramsSummaryMap()`) and the creation of the help messages.

```groovy
validation.parametersSchema = "path/to/schema.json" // default "nextflow_schema.json"
```

This option can either be a path relative to the root of the pipeline directory or a full path to the JSON schema (Be wary to not use hardcoded local paths to ensure your pipeline will keep working on other systems)

## monochromeLogs

This option can be used to turn of the colored logs from nf-validation. This can be useful if you run a Nextflow pipeline in an environment that doesn't support colored logging.
Expand All @@ -46,7 +56,11 @@ validation.failUnrecognisedParams = <true|false> // default: false

## showHiddenParams

By default the parameters, that have the `"hidden": true` annotation in the JSON schema, will not be shown in the help message created by `paramsHelp()`. Turning on this option will make sure the hidden parameters are also shown.
!!! deprecated

This configuration option has been <b>deprecated</b> since v2.1.0. Please use `validation.help.showHidden` instead.

By default the parameters, that have the `"hidden": true` annotation in the JSON schema, will not be shown in the help message. Turning on this option will make sure the hidden parameters are also shown.

```groovy
validation.showHiddenParams = <true|false> // default: false
Expand All @@ -71,3 +85,93 @@ This option does exactly the same as `validation.ignoreParams`, but provides pip
```groovy
validation.defaultIgnoreParams = ["param1", "param2"] // default: []
```

## help

The `validation.help` config scope can be used to configure the creation of the help message.

This scope contains the following options:

### enabled

This option is used to enable the creation of the help message when the help parameters are used in the `nextflow run` command.

```groovy
validation.help.enabled = true // default: false
```

### shortParameter

This option can be used to change the `--help` parameter to another parameter. This parameter will print out the help message with all top level parameters.

```groovy
validation.help.shortParameter = "giveMeHelp" // default: "help"
```

`--giveMeHelp` will now display the help message instead of `--help` for this example. This parameter will print out the expanded help message.
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

### fullParameter

This option can be used to change the `--helpFull` parameter to another parameter.

```groovy
validation.help.shortParameter = "giveMeHelpFull" // default: "helpFull"
```

`--giveMeHelpFull` will now display the expanded help message instead of `--helpFull` for this example.

### showHiddenParameter

This option can be used to change the `--showHidden` parameter to another parameter. This parameter tells the plugin to also include the hidden parameters into the help message.

```groovy
validation.help.showHiddenParameter = "showMeThoseHiddenParams" // default: "showHidden"
```

`--showHidden` will now make sure hidden parameters will be shown instead of `--showHidden` for this example.
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

### showHidden

By default the parameters, that have the `"hidden": true` annotation in the JSON schema, will not be shown in the help message. Turning on this option will make sure the hidden parameters are also shown.

```groovy
validation.help.showHidden = <true|false> // default: false
```

### beforeText

!!! example "This option does not affect `paramsHelp()`"
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

Any string provided to this option will printed before the help message.

```groovy
validation.help.beforeText = "Running pipeline version 1.0" // default: ""
```

### command

!!! example "This option does not affect `paramsHelp()`"
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

This option can be used to add an example command to the help message. This will be printed after the `beforeText` and before the help message.

```groovy
validation.help.command = "nextflow run main.nf --input samplesheet.csv --outdir output" // default: ""
```

This example will print the following message:

```bash
Typical pipeline command:

nextflow run main.nf --input samplesheet.csv --outdir output
```

### afterText

!!! example "This option does not affect `paramsHelp()`"
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

Any string provided to this option will printed after the help message.
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

```groovy
validation.help.afterText = "Please cite the pipeline owners when using this pipeline" // default: ""
```
28 changes: 26 additions & 2 deletions docs/contributing/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,33 @@ To compile and run the tests use the following command:
./gradlew check
```

## Launch it with Nextflow
## Launch it with installed Nextflow

To test with Nextflow for development purpose:
!!! warning

This method will add the development version of the plugin to your Nextflow plugins
Take care when using this method and make sure that you are never using a
development version to run real pipelines.
You can delete all `nf-schema` versions using this command:
```bash
rm -rf ~/.nextflow/plugins/nf-schema*
```

Install the current version of the plugin in your `.nextflow/plugins` folder
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

```bash
make install
```

Update or add the nf-schema plugin with the installed version in your test pipeline

```groovy title="nextflow.config"
plugins {
id 'nf-schema@x.y.z'
}
```

## Launch it with a local version of Nextflow

Clone the Nextflow repo into a sibling directory

Expand Down
27 changes: 27 additions & 0 deletions docs/migration_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Following list shows the major breaking changes introduced in nf-schema:
3. The `unique` keyword for samplesheet schemas has been removed. Please use [`uniqueItems`](https://json-schema.org/understanding-json-schema/reference/array#uniqueItems) or [`uniqueEntries`](nextflow_schema/nextflow_schema_specification.md#uniqueentries) now instead.
4. The `dependentRequired` keyword now works as it's supposed to work in JSON schema. See [`dependentRequired`](https://json-schema.org/understanding-json-schema/reference/conditionals#dependentRequired) for more information.
5. All configuration parameters have been converted to Nextflow configuration options. See [Updating configuration](#updating-configuration) for more information.
6. Help messages are now created automatically instead of using the `paramsHelp()` function. (v2.1.0 feature)

A full list of changes can be found in the [changelog](https://github.com/nextflow-io/nf-schema/blob/master/CHANGELOG.md).

Expand Down Expand Up @@ -263,3 +264,29 @@ When you use `dependentRequired` in your schemas, you should update it like this
}
}
```

### Updating the help message

!!! example "v2.1.0 feature"

The creation of the help message now needs to be enabled in the configuration file. Using `--help` or `--helpFull` will automatically print the help message and stop the pipeline execution. `paramsHelp()` is still available in `nf-schema` and can still be used like before. This could be helpful to print the help message in specific cases. Mind that this function now automatically emits a deprecation warning. This warning can be disabled using the `hideWarning:true` option of the function.

=== "nf-validation"

```groovy title="main.nf"
if (params.help) {
log.info paramsHelp("nextflow run my_pipeline --input input_file.csv")
exit 0
}
```

=== "nf-schema"

```groovy title="nextflow.config"
validation {
help {
enabled: true
command: "nextflow run my_pipeline --input input_file.csv"
}
}
```
28 changes: 21 additions & 7 deletions docs/nextflow_schema/nextflow_schema_specification.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,7 @@ However, they will be displayed as ungrouped in tools working off the schema.

## Nested parameters

!!! warning "Warning (TLDR;)"

Although the JSON Schema allows schema objects (eg.` params.foo.bar = "baz"`) or arrays, this is not supported by this plugin.
!!! example "New feature in v2.1.0"

Nextflow config allows parameters to be nested as objects, for example:

Expand All @@ -102,12 +100,28 @@ or on the CLI:
nextflow run <pipeline> --foo.bar "baz"
```

But - the current implementations of the Nextflow _schema_ do not (for now).
Nested parameters can be specified in the schema by adding a `properties` keyword to the root parameters:

This was done as a conscious decision when the code was first written, to try to reduce complexity.
```json
{
"type": "object",
"properties": {
"nested": {
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved
// Annotation for the --nested parameter
"type": "object", // Parameters that contain subparameters need to have the "object" type
"properties": {
// Add other parameters in here
"deep": {
// Annotation for the --nested.deep parameter
"type": "string"
}
}
}
}
}
```

It would be great to implement this at some point - there is a GitHub issue to track the feature request here:
[`nf-core/tools#1554`](https://github.com/nf-core/tools/issues/1554). Contributions welcome!
There is no limit to how deeply nested parameters can be. Mind however that deeply nested parameters are not that user friendly and will create some very ugly help messages. It's adviced to not go deeper than two levels of nesting, but it's entirely possible to go deeper.
nvnieuwk marked this conversation as resolved.
Show resolved Hide resolved

## Required parameters

Expand Down
Loading
Loading