basic cylc 8 syntax checker to assist upgrade

[oliver-sanders]

Cylc gives deprecation warnings which is helpful.

Unfortunately, because of Jinja2 it is likely that there are bits of the config that Cylc will not load:

#!Jinja2
[scheduling]
    [[dependencies]]
        [[[P1D]]]
            graph = ...
{% if COND %}
         [[[PT6H]]]
             graph = ...
{% endif %}

[runtime]
    [[x]]
        ...
{% include "site/" + SITE + '.rc' with context %}

With Cylc 7 we had a lot of cases where suites would break after users turned on an option that hadn't been used in a while.

To make life easier we could consider providing a simple script that picks out the most obvious bits of invalid syntax without parsing the file.

Example:

Here's a quick and simple approach using sed that adds comments after lines that contain Cylc 7 syntax.

# 728gen.py
UPG = {                                                              
    '[[[remote]]]': 'flow.cylc[runtime][<namespace>]platform',      
    'graph.*=': 'flow.cylc[scheduling][graph]<recurrence>',    
}                                                    
                                                     
REGEX_CHARS = ['[', ']', '(', ')']                   
                                                     
                                                     
def escape(pattern):                                 
    for char in REGEX_CHARS:                         
        pattern = pattern.replace(char, rf'\{char}')    
    return pattern                                   
                                                     
                                                     
def main(write):                                     
    for pattern, msg in UPG.items():                 
        write(rf's|^\([^#]*{escape(pattern)}.*\)$|\1  # <= {msg}|')                  
                                                                     
                                                                     
if __name__ == '__main__':                                           
    main(write=print)  

$ python 728gen.py > 728.sed
$ sed -f 727.sed suite.rc
[scheduling]
    [[dependencies]]
        [[[R1]]]
            graph = """  # <= flow.cylc[scheduling][graph]<recurrence>
                foo
            """

[runtime]
    [[foo]]
        [[[remote]]]  # <= flow.cylc[runtime][<namespace>]platform
            host = foo

Pull requests welcome!

[hjoliver]

Unfortunately, because of Jinja2 it is likely that there are bits of the config that Cylc will not load:

I was just about to add a warning to the 7-8 migration docs about this. I'll hold off in case we decide to go with this approach.

[wxtim]

I was just about to add a warning to the 7-8 migration docs about this. I'll hold off in case we decide to go with this approach.

I think the warning in the documentation is valid either way.

[oliver-sanders]

We think this would be a good idea, possibly fronted by a cylc command. Can be implemented via find -exec sed or manually via Python.

This becomes important when people start upgrading workflows out of Cylc 7 compatibility mode.