Cylc 728 (with linter)

[wxtim]

Work in Progress

Posted as a PR for discussion on whether the approach seems OK - it's slightly over-engineered on the basis that it could become a Cylc Linter with very little work.

These changes close #4519

Requirements check-list

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg and conda-environment.yml.
• Appropriate tests are included (unit and/or functional).
• Appropriate change log entry included.
• (master branch) I have opened a documentation PR at Document cylc lint cylc-doc#480.

[MetRonnie]

Looks good 👍 only thing is we should probably exclude the log directory

[oliver-sanders]

Would probably call the command 728 rather than linter since it's upgrading not linting (we had a cylc 526 command in the past). Would be good to leave the door open to a linter one day.

[MetRonnie]

I have tried this on

[cylc]
    UTC mode = True
[scheduling]
    initial cycle point = 2000-01-01T00Z
    final cycle point = 2000-01-04T00Z
    [[dependencies]]
        [[[PT3H]]]
            graph = """
                get_observations_belmullet & get_observations_camborne & get_observations_heathrow & get_observations_shetland => consolidate_observations
            """

expecting it to pick up on [cylc] and graph =, but instead it just says

Checked cylc7-datetime-cycling/suite.rc and found 0 issues.

[wxtim]

I have tried this on

[cylc]
    UTC mode = True
[scheduling]
    initial cycle point = 2000-01-01T00Z
    final cycle point = 2000-01-04T00Z
    [[dependencies]]
        [[[PT3H]]]
            graph = """
                get_observations_belmullet & get_observations_camborne & get_observations_heathrow & get_observations_shetland => consolidate_observations
            """

expecting it to pick up on [cylc] and graph =, but instead it just says

Checked cylc7-datetime-cycling/suite.rc and found 0 issues.

I cannot replicate this - does it still happen?

I get

> cylc 728 
[001: 7-to-8]/net/home/h02/tpilling/cylc-src/temp/flow.cylc: 0: section ``[cylc]`` is now called ``[scheduler]``.
[021: 7-to-8]/net/home/h02/tpilling/cylc-src/temp/flow.cylc: 5: ``[dependencies]`` is deprecated.
[022: 7-to-8]/net/home/h02/tpilling/cylc-src/temp/flow.cylc: 7: ``[cycle point]graph =`` is deprecated, use ``cycle point = <graph>``
Checked /net/home/h02/tpilling/cylc-src/temp and found 3 issues.

[wxtim]

@MetRonnie - I showed @oliver-sanders the fact that I've effectively already written a linter and he got a bit excited and made a fair few suggestions. I've implemented a couple so this PR has changed again.

[hjoliver]

Partial review done @wxtim but there's a bunch of spurious changes that already exist on master - might be a rebase required?

[wxtim]

@hjoliver - I think that I must have botched a rebase - I think that's now fixéd.

[MetRonnie]

Linting works, however the line numbers are 1 lower than they should be

[oliver-sanders]

Linting works, however the line numbers are 1 lower than they should be

This may be due to the shebang line? If so, note this may or may not be present.

[wxtim]

Linting works, however the line numbers are 1 lower than they should be

required a start=1 kwarg to an enumerate()

[MetRonnie]

@wxtim Flake8 failure (and while you're at it could you squash down these Update cylc/flow/scripts/lint.py commits?)

[MetRonnie]

I'm getting spurious changes coming through in f171c72 (undoing recent commits on master). This is my effort at rebasing: https://github.com/MetRonnie/cylc-flow/tree/tmp. If you're happy with it could you reset your branch to the tip of that?

(Converting to draft so as not to merge with these reversions)

[MetRonnie]

Now it is running 728 checks on suite.rc workflows

[oliver-sanders]

Found two small issues (sorry):

1. When linting multiple workflows the number of reported issues is carried over from previous workflows to future ones.
2. The [runtime][<namespace>][environment] section is being erroneously flagged as the obsoleted [cylc][environment] section.

[MetRonnie]

Formatting issue

$ cylc lint simple-c7-datetime/
Checked ~/cylc-run/simple-c7-datetime against style checksFound no issues.
$ #                                                       ^ missing punctuation

[MetRonnie]

"checksrules"

Checked {target} against {check_names} checksrules and found {count} issue(s).

[oliver-sanders]

Tested, happy my review comments have been addressed.

(please squash this one when you merge 😁)

[MetRonnie]

🎇🎊🎉