Customising your Nektool

I've cookiecuttered the template, now what?

Adding and editing command line options, and adding new subcommands is almost identical to Snaketool. First, install it as a python module cd my_nektool && pip install -e . so that you can easily run, rerun, rererun while developing your launcher. Next, build your pipeline in the workflow dir. The example simply uses a workflow.nf, nextflow.config, and params.yaml, but you should add directories for environments, rules, etc. following NextFlow best practices and conventions where possible.

Finally, open up __main__.py in your favourite Python IDE and get to work!

Recommended reading

Familiarising yourself with the Click command line interface for Python will help you through this guide.

Adding common command line options

If you have command line options that you want to use in more than one subcommand, you can add them to common_options(). Let's add an option to define a temporary directory

 def common_options(func):
     """Common options decorator for use with click commands."""
     options = [
         click.option('--paramsfile', default='params.yaml', help='Custom params file', show_default=True),
         click.option('--configfile', default='nextflow.config', help='Custom config file', show_default=True),
+        click.option('--temp', help='Directory for temporary files', type=click.Path(),
+                     default='my_nektool.temp', show_default=True),
         click.option('--threads', help='Number of threads to use', default=1, show_default=True),
         click.option('--use-conda/--no-use-conda', default=True, help='Use conda for NextFlow rules',
                      show_default=True),
         click.option('--conda-frontend',
                      type=click.Choice(['mamba', 'conda'], case_sensitive=True),
                      default='mamba', help='Specify Conda frontend', show_default=True),
         click.option('--conda-prefix', default=nek_base(os.path.join('workflow', 'conda')),
                      help='Custom conda env directory', type=click.Path(), show_default=False),
         click.argument('nextflow_args', nargs=-1)
     ]
     for option in reversed(options):
         func = option(func)
     return func

Customising subcommands

The subcommand run() launches the main pipeline workflow.nf. This also demonstrates all the available options when calling the run_nextflow() function.

Most of the customisation you will probably want to do will be to add command line arguments that will be added to the configuration.

• Add new args as click options
• define them when calling the subcommand script
• add them to the merge_config dictionary.

That's it! The new options will be available within the Snakemake config dictionary.

 @click.command(epilog=help_msg_extra, context_settings=dict(help_option_names=["-h", "--help"], ignore_unknown_options=True))
 @click.option('--input', '_input', help='Input file/directory', type=str, required=True)
+@click.option('--search', type=click.Choice(['fast', 'slow'], case_sensitive=False), default='fast', help='Search setting', show_default=True)
 @common_options
-def run(_input, **kwargs):
+def run(_input, temp, search, **kwargs):
     """Run My Nektool"""
     # Config to add or update in configfile
     merge_config = {
         'input': _input,
+        'search': search,
+        'temp': temp,
     }

     # run!
     run_nextflow(
         nextfile_path=nek_base(os.path.join('workflow', 'workflow.nf')),
         merge_config=merge_config,
         **kwargs
     )

Adding new subcommands

Adding new subcommands is relatively easy. Say we have a super simple NextFlow script for installing the databases. Let's make it available with an install subcommand, e.g. my_nektool install .... Use run() as a template and strip out what you dont want. We will only keep the common options for running NextFlow.

• Create the installation NextFlow script: workflow/install.nf
• Create new subcommand function (use 'run()' as a template): install()
• Update the function doc string, which will become the help message for the subcommand

 @click.command(epilog=help_msg_extra, context_settings=dict(help_option_names=["-h", "--help"], ignore_unknown_options=True))
-@click.option('--input', '_input', help='Input file/directory', type=str, required=True)
-@click.option('--search', type=click.Choice(['fast', 'slow'], case_sensitive=False), default='fast', help='Search setting', show_default=True)
 @common_options
-def run(_input, temp, search, **kwargs):
+def install(**kwargs):
-    """Run My Nektool"""
+    """Install databases"""
     # Config to add or update in configfile
-    merge_config = {
-        'input': _input,
-        'search': search,
-        'temp': temp,
-    }

     # run!
     run_nextflow(
-        nextfile_path=nek_base(os.path.join('workflow', 'workflow.nf')),
+        nextfile_path=nek_base(os.path.join('workflow', 'install.nf')),
-        merge_config=merge_config,
         **kwargs
     )

Lastly, add this function name to click's list of commands. Note the order of commands is preserved in the click help message.

 cli.add_command(run)
+cli.add_command(install)
 cli.add_command(config)