Adapt to module syntax of upcoming Snakemake 6.0

[johanneskoester]

The upcoming Snakemake 6.0 (release today or Monday) will have a new module system and syntax: snakemake/snakemake#888

Examples:

configfile: "config/config.yaml"


module test:
    snakefile:
        "module-test/Snakefile"
    config:
        config
    replace_prefix: 
        {"results/": "results/testmodule/"}


use rule * from test

configfile: "config.yaml"


module bwa_mapping:
    meta_wrapper: "0.72.0/meta/bio/bwa_mapping"


use rule * from bwa_mapping


def get_input(wildcards):
    return config["samples"][wildcards.sample]


use rule bwa_mem from bwa_mapping with:
    input:
        get_input

module test:
    snakefile: "module-test/Snakefile"


use rule * from test as test_*


use rule a from test as test_* with:
    output:
        "test2.out"
    params:
        test=2

rule a:
    output:
        "test.out"
    shell:
        "echo test > {output}"


use rule a as b with:
    output:
        "test2.out"

updated EBNF:

snakemake    = statement | rule | include | workdir | module | configfile | container
    rule         = "rule" (identifier | "") ":" ruleparams
    include      = "include:" stringliteral
    workdir      = "workdir:" stringliteral
    module       = "module" identifier ":" moduleparams
    configfile   = "configfile" ":" stringliteral
    userule      = "use" "rule" (identifier | "*") "from" identifier ["as" identifier] ["with" ":" norunparams]
    ni           = NEWLINE INDENT
    norunparams  = [ni input] [ni output] [ni params] [ni message] [ni threads] [ni resources] [ni log] [ni conda] [ni container] [ni benchmark] [ni cache]
    ruleparams   = norunparams [ni (run | shell | script | notebook)] NEWLINE snakemake
    input        = "input" ":" parameter_list
    output       = "output" ":" parameter_list
    params       = "params" ":" parameter_list
    log          = "log" ":" parameter_list
    benchmark    = "benchmark" ":" statement
    cache        = "cache" ":" bool
    message      = "message" ":" stringliteral
    threads      = "threads" ":" integer
    resources    = "resources" ":" parameter_list
    version      = "version" ":" statement
    conda        = "conda" ":" stringliteral
    container    = "container" ":" stringliteral
    run          = "run" ":" ni statement
    shell        = "shell" ":" stringliteral
    script       = "script" ":" stringliteral
    notebook     = "notebook" ":" stringliteral
    moduleparams = [ni snakefile] [ni metawrapper] [ni config] [ni skipval]
    snakefile    = "snakefile" ":" stringliteral
    metawrapper  = "meta_wrapper" ":" stringliteral
    config       = "config" ":" stringliteral
    skipval      = "skip_validation" ":" stringliteral

[johanneskoester]

Do you think you will be able to release a snakefmt update at the same time?
Sorry for the late notice, I completely forgot to let you know in advance...

[bricoletc]

Cool! Can look at this for monday, but can't guarantee anything will be ready by then. Will work to do this as soon as possible, most likely inside of a week.

[johanneskoester]

Great, thanks! I'll released now, but when announcing I will just tell people that they need to be patient a few more days with regards to snakefmt.
In case it helps, here is the snippet for the module part:

class SnakeModule(Vocabulary):
    spec = dict(
        snakefile=Grammar(None, SingleParam),
        config=Grammar(None, SingleParam),
        skip_validation=Grammar(None, SingleParam),
        meta_wrapper=Grammar(None, SingleParam),
        replace_prefix=Grammar(None, SingleParam),
    )

I was already about to create a PR myself, but it appeared to me I need to know more about snakefmts grammar implementation in order to be able to properly encode the new use rule statement.

[bricoletc]

This looks all done.
@johanneskoester I wonder have you ever considered using a parser generator, eg lark, to go from snakemake EBNF to parsing?
I'm trying to think in terms of avoiding maintaining two different parsers in snakemake and snakefmt, to make it easier for snakemake-aware coders to update snakefmt.