Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cylc 728 (with linter) #4900

Merged
merged 59 commits into from
Jul 27, 2022
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
b8377ad
cylc linter
wxtim Jul 4, 2022
e0df94e
fixed: non indents on jinja2 OK
wxtim Jul 4, 2022
ae550e4
stuff
wxtim Jul 4, 2022
947cd72
add documentation for API for pre and post install plugins.
wxtim Jul 5, 2022
1d023ad
Update CHANGES.md
wxtim Jul 5, 2022
b147c42
Merge branch 'master' into cylc-linter
wxtim Jul 5, 2022
9a35f32
Merge branch 'master' into cylc-linter
wxtim Jul 5, 2022
6358a21
remove file added by mistake
wxtim Jul 5, 2022
7407bf6
undo mistaken reversions
wxtim Jul 5, 2022
719404b
re-instated --reference mode; made doc not RST
wxtim Jul 5, 2022
94621aa
added automated scraping of parsec upgraders
wxtim Jul 5, 2022
d79bc6d
wip - scraping obseletions
wxtim Jul 7, 2022
4631e80
stuff
wxtim Jul 10, 2022
34ae482
re-written linter to use Cylc workflow config
wxtim Jul 10, 2022
c79370f
updated tests to reflect all changes
wxtim Jul 10, 2022
50243d5
fix functional test
wxtim Jul 11, 2022
079892b
add test for config upgrader script
wxtim Jul 11, 2022
5cdcb76
flake8
wxtim Jul 11, 2022
ce88cc0
Update cylc/flow/scripts/lint.py
wxtim Jul 18, 2022
0ea6523
fix some issues
wxtim Jul 18, 2022
c223549
Merge branch 'cylc-linter' of github.com:wxtim/cylc into cylc-linter
wxtim Jul 18, 2022
ca6db34
Apply suggestions from code review
wxtim Jul 18, 2022
578094d
uncommented a test
wxtim Jul 18, 2022
4f79272
put all sections in []
wxtim Jul 18, 2022
b294624
added testing for prettify config function
wxtim Jul 18, 2022
fc2ee9f
fix bug @ronnie had
wxtim Jul 19, 2022
d36ca92
Added manual upgrade checks for items with complex upgraders.
wxtim Jul 19, 2022
c0df60c
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
4a19b09
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
bacada2
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
85f83c5
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
1226f56
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
134d752
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
d0f13df
Update cylc/flow/scripts/lint.py
wxtim Jul 21, 2022
165f6e2
Merge branch 'master' into cylc-linter
wxtim Jul 21, 2022
350a287
Make the linter default behaviour be linting
wxtim Jul 21, 2022
01e39aa
fix regex
wxtim Jul 21, 2022
d440b61
fix flake8 bug which crept into master
wxtim Jul 21, 2022
5b9b381
updated functional test
wxtim Jul 21, 2022
ebb8d6c
fixed broken line
wxtim Jul 21, 2022
cbb459b
merge
wxtim Jul 21, 2022
8562293
Merge branch 'master' into cylc-linter
wxtim Jul 22, 2022
a3964f0
cylc lint: review feedback
oliver-sanders Jul 25, 2022
b1596b1
Merge pull request #44 from oliver-sanders/cylc-lint
wxtim Jul 25, 2022
8b8a9e8
Apply suggestions from code review
wxtim Jul 26, 2022
44cc3e6
Merge branch 'master' into cylc-linter
wxtim Jul 26, 2022
9ee971d
fix bug
wxtim Jul 26, 2022
ffa7138
fix problems @osanders found on review
wxtim Jul 26, 2022
31d6799
fix unit tests
wxtim Jul 26, 2022
bef898d
fixed functional tests
wxtim Jul 26, 2022
3692ca7
SPAG
wxtim Jul 27, 2022
f403bcf
made __doc__ rst to make autodoc work nicely.
wxtim Jul 27, 2022
f9a2c56
fix test
wxtim Jul 27, 2022
fea7250
Apply suggestions from code review
wxtim Jul 27, 2022
7722995
Apply suggestions from code review
wxtim Jul 27, 2022
ff83770
fixed review bug -
wxtim Jul 27, 2022
8229168
fix
wxtim Jul 27, 2022
f079b35
Update cylc/flow/scripts/lint.py
wxtim Jul 27, 2022
1005f08
nice plural
wxtim Jul 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,10 @@ Jinja2 used by Cylc from 2.11 to 3.0.
[#4896](https://github.com/cylc/cylc-flow/pull/4896) - Allow the setting of
default job runner directives for platforms.

[#4900](https://github.com/cylc/cylc-flow/pull/4900) - Added a command to assist
with upgrading Cylc 7 workflows to Cylc 8: Try `cylc lint <workflow-dir>`.


### Fixes

[#4984](https://github.com/cylc/cylc-flow/pull/4984) -
Expand Down
14 changes: 10 additions & 4 deletions cylc/flow/cfgspec/workflow.py
Original file line number Diff line number Diff line change
Expand Up @@ -1704,7 +1704,7 @@ def upg(cfg, descr):
u.obsolete('7.8.1', ['cylc', 'events', 'reset inactivity timer'])
u.obsolete('8.0.0', ['cylc', 'force run mode'])
u.obsolete('7.8.1', ['runtime', '__MANY__', 'events', 'reset timer'])
u.obsolete('8.0.0', ['cylc', 'authentication'])
u.obsolete('8.0.0', ['cylc', 'authentication'], is_section=True)
u.obsolete('8.0.0', ['cylc', 'include at start-up'])
u.obsolete('8.0.0', ['cylc', 'exclude at start-up'])
u.obsolete('8.0.0', ['cylc', 'log resolved dependencies'])
Expand All @@ -1721,13 +1721,13 @@ def upg(cfg, descr):
)
u.obsolete('8.0.0', ['cylc', 'abort if any task fails'])
u.obsolete('8.0.0', ['cylc', 'disable automatic shutdown'])
u.obsolete('8.0.0', ['cylc', 'environment'])
u.obsolete('8.0.0', ['cylc', 'environment'], is_section=True)
u.obsolete('8.0.0', ['cylc', 'reference test'])
u.obsolete(
'8.0.0',
['cylc', 'simulation', 'disable suite event handlers'])
u.obsolete('8.0.0', ['cylc', 'simulation'])
u.obsolete('8.0.0', ['visualization'])
u.obsolete('8.0.0', ['cylc', 'simulation'], is_section=True)
u.obsolete('8.0.0', ['visualization'], is_section=True)
u.obsolete('8.0.0', ['scheduling', 'spawn to max active cycle points']),
u.deprecate(
'8.0.0',
Expand All @@ -1740,12 +1740,14 @@ def upg(cfg, descr):
['cylc', 'parameters'],
['task parameters'],
silent=cylc.flow.flags.cylc7_back_compat,
is_section=True,
)
u.deprecate(
'8.0.0',
['cylc', 'parameter templates'],
['task parameters', 'templates'],
silent=cylc.flow.flags.cylc7_back_compat,
is_section=True,
)
# Whole workflow task mail settings
for mail_setting in ['to', 'from', 'footer']:
Expand Down Expand Up @@ -1802,6 +1804,7 @@ def upg(cfg, descr):
['runtime', '__MANY__', 'suite state polling'],
['runtime', '__MANY__', 'workflow state polling'],
silent=cylc.flow.flags.cylc7_back_compat,
is_section=True
)

for job_setting in [
Expand Down Expand Up @@ -1875,6 +1878,7 @@ def upg(cfg, descr):
['cylc'],
['scheduler'],
silent=cylc.flow.flags.cylc7_back_compat,
is_section=True,
)
u.upgrade()

Expand All @@ -1884,6 +1888,8 @@ def upg(cfg, descr):
warn_about_depr_platform(cfg)
warn_about_depr_event_handler_tmpl(cfg)

return u


def upgrade_graph_section(cfg: Dict[str, Any], descr: str) -> None:
"""Upgrade Cylc 7 `[scheduling][dependencies][X]graph` format to
Expand Down
23 changes: 18 additions & 5 deletions cylc/flow/parsec/upgrade.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,10 @@ def __init__(self, cfg, descr):
# upgrades must be ordered in case several act on the same item
self.upgrades = OrderedDict()

def deprecate(self, vn, oldkeys, newkeys=None, cvtr=None, silent=False):
def deprecate(
self, vn, oldkeys, newkeys=None,
cvtr=None, silent=False, is_section=False,
):
"""Replace a deprecated key from a config
Args:
vn (str):
Expand All @@ -62,17 +65,22 @@ def deprecate(self, vn, oldkeys, newkeys=None, cvtr=None, silent=False):
cvtr (cylc.flow.parsec.upgrade.Converter):
Converter object containing a conversion function and a
description of that function.
silent:
silent (bool):
Set silent mode for this upgrade.
is_section (bool):
Is a section heading.
"""
if vn not in self.upgrades:
self.upgrades[vn] = []
if cvtr is None:
cvtr = converter(lambda x: x, "value unchanged") # identity
self.upgrades[vn].append(
{'old': oldkeys, 'new': newkeys, 'cvt': cvtr, 'silent': silent})
{
'old': oldkeys, 'new': newkeys, 'cvt': cvtr,
'silent': silent, 'is_section': is_section
})

def obsolete(self, vn, oldkeys, silent=False):
def obsolete(self, vn, oldkeys, silent=False, is_section=False):
"""Remove an obsolete key from a config
Args:
vn (str):
Expand All @@ -81,12 +89,17 @@ def obsolete(self, vn, oldkeys, silent=False):
Path within config to be removed.
silent:
Set silent mode for this upgrade.
is_section (bool):
Is a section heading.
"""
if vn not in self.upgrades:
self.upgrades[vn] = []
cvtr = converter(lambda x: x, "DELETED (OBSOLETE)") # identity
self.upgrades[vn].append(
{'old': oldkeys, 'new': None, 'cvt': cvtr, 'silent': silent})
{
'old': oldkeys, 'new': None, 'cvt': cvtr, 'silent': silent,
'is_section': is_section
})

def get_item(self, keys):
item = self.cfg
Expand Down
Loading