Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

syntax: add reference syntax files for testing lexers #2916

Merged
merged 2 commits into from
Feb 6, 2019
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 115 additions & 0 deletions etc/syntax/reference-files/suite.rc
Original file line number Diff line number Diff line change
@@ -0,0 +1,115 @@
# SYNTAX numbering matches the table in https://github.com/cylc/cylc/issues/2752

# SYNTAX: 1.

# SYNTAX: 1.1
# keys and values should display differently
# sections should be highlighted
[a]
b = c
[[d]]
-j = k # keys can have hyphens
j j = k # and spaces
g = h=i # values can have equals
e = "f" # and quotes
e = "${HOME} `$HOME`" # and funny characters
[[l, m, n]] # sections can be contain commas
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
[[[o]]] # sections can be deeply nested

# SYNTAX: 1.2
l=m # no comment

sadielbartholomew marked this conversation as resolved.
Show resolved Hide resolved
# SYNTAX: 1.3
# display should be consistant throught multi-line string
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
n = """foo"""
n = """
foo
bar
baz
"""

# SYNTAX: 1.4
# highlighting should be consistent with the following examples
%include bar
%include "bar/baz"
%include "baz/bo ol"


# SYNTAX: 2.
graph = """
# settings should not be highlighted here
a = b

# SYNTAX: 2.1
# tasks and dependency arrows should display differently
foo => bar
foo=>bar

# chaining should work in-line and across lines ending in `=>` or `|`
foo => bar => baz
foo => bar =>
pub => baz
foo => \
bar => \
pub

# SYNTAX: 2.2
# comment highlighting should be un-affected
foo => bar # pub
foo => bar # pub => baz

# SYNTAX: 2.3
# &|() should be highlighted
# !foo should be highlighted
# ! foo should NOT be highlighted
foo | bar & baz => pub & qux
(foo | bar) & baz => (pub & qux)
foo => !foo & bar
! foo => bar

# SYNTAX: 2.4
# text inside <> should be highlighted
# optionally suite / task could be highlighted
<other.suite::foo> => foo
foo & <other.suite::task_bar> => baz

# SYNTAX: 2.5
# @x should be highlighted
@wall_clock => foo
foo & @myxtrig1 => bar

# SYNTAX: 2.6
# text inside <> should be highlighted
foo<x> => bar<x>
<x> => <y>

# SYNTAX: 2.7
# text inside <> should be highlighted
# optionally +- could be highlighted
<x-1> => <x+1>
<x - 1> => <x + 1>

# SYNTAX: 2.8
# text inside <> should be highlighted
# optionally = could be highlighted
a<x=1> => a<x=2>
a<x = 1> => a<x = 2>

# SYNTAX: 2.9
# qualifier should be highlighted, whole and nothing but
foo:succeed => bar
pub & foo:fail => bar
FOO:succeed-all => bar
foo:custom_a => bar

# SYNTAX: 2.10
# inter-cycle dep should be highlighted
foo[P1D] => bar
foo[-P1D] => bar
foo[-P1Y1M1DT1H1M1S] => bar
foo[^] => bar
foo[123] => bar # optionally invalid offsets should display as an error
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
"""

# graph strings should not highlight here, this should display as a setting:
foo => bar => baz => quz
75 changes: 75 additions & 0 deletions etc/syntax/reference-files/suite_empy.rc
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
#!empy
# ^ SYNTAX:3.1

# SYNTAX numbering matches the table in https://github.com/cylc/cylc/issues/2752

# SYNTAX: 3.2
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
# templating code should be displayed diffently to cylc code
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
@# no comment # should display as a comment
@foo
@{foo="bar"}
@{ foo = "bar" }
@{ foo = "bar"; bar = "baz" }
@{ foo = '%s' % 'bar' }
@{ set foo = {"1": 2, "3", []} } # highlight to the last }
@{ set foo = foo.bar(1, 2, 3) } # highlight to the last }
@{ foo_bar }
@{foo_bar}
@{ foo.bar(1, 2, 3, []) } # highlight to the last }
@[if x]a = b@[end if] # highlight all but "a = b"
@[if x.y[]]a = b@[end if] # highlight all but "a = b"
a = b
# SYNTAX 3.3
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
# non-exhaustive list of advanced empy syntax
@# no comment
@?NewContextName
@!100
invisible@ whitespace # the @ character should be highlighted
foo = @\0 bar @\xHH # @\0 and @\xHH should both be highlighted
foo = @@ # the @@ should NOT be highlighted
@"foo" # this should be highlighted the same as @foo?
@( x ? "true" ! "false )
@os.environ
@`x`
@:2 + 2:
:%title "Cylc"
**'@<CONTENTS>'** # ???
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
@[def f(x, y=z, **kwargs)] @[end def]
@[try] @[except]

# SYNTAX 3./1.1
# keys, values and sections outside of the templating code should display
# as normal
a = @b
@a = b
a = @( b + 1 )
a = @( b_b | c.d('1', "2", 3) )
[[@a}]]
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
d = e # check highlighting of this line un-affected by section
[[@( a_a | c.d('1', "2", 3) )]]
d = e # check highlighting of this line un-affected by section
[[@( "%(foo)s" % {"foo": "abc"} )]]
d = e # check highlighting of this line un-affected by section

# SYNTAX 3./1.2
# comments outside of the templating code should display as normal
@( foo ) # foo
@{ foo } # foo
@[ foo ] # foo

# syntax 3./1.3
# tasks and deps outside of the templating code should display as normal
graph = """
foo => @( b_az.pub('1', "2", 3) ) => qux =>
@[ if BAR ]
bar => pub
@[ else ]
baz => pub
@[ endif ]
"""

# syntax 3./2.5
# clashing xtrigger and empy syntax should somehow be handled
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well anticipated!

In general, implementing lexers supporting all the listed syntactical elements cleanly & consistently will be rather tricky I think, for reasons such as this i.e. given different interpretations under the suite.rc mode & the two templating engines.

graph = """
@wall_clock => @@bar => baz # note empy evaluates @@ as @
"""
70 changes: 70 additions & 0 deletions etc/syntax/reference-files/suite_jinja2.rc
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
#!Jinja2
# ^ SYNTAX:3.1

# SYNTAX numbering matches the table in https://github.com/cylc/cylc/issues/2752

# SYNTAX: 3.2
# templating code should be displayed diffently to cylc code
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
{# no comment #} # should display as a comment
{% set foo = "bar" %}
{% set foo = '%s' % 'bar' %}
{% set foo = {"1": 2, "3", []} %}
{% set foo = foo.bar(1, 2, 3) %}
{{ foo_bar }}
{{ foo.bar(1, 2, 3) }}
{{ foo | bar(1, 2, 3) }}
{% include 'foo' %}
{% include ['foo', 'bar'] %}
{% import 'foo' as foo %}
{% from 'foo' import foo as a, b %}

# SYNTAX 3.3
# multi-line templating code should be consistent with single-line code
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps to convey that all four examples below are meant to display consistently as a unit, though only the top two are commented as such at the moment, add instead one line at the top here similar to # for each case, all three lines should display as a single unit.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kinda depends how template markup is handled, for instance the syntax highlighter might wish to highlight keywords or literals differently.

I've updated the comment to make extra clear that whatever the behaviour implemented by the lexer it should be consistent no matter how many lines they are spread over.

{#
no comment
#} # all three lines should display as a single comment
{# a
b
c #} # all three lines should display as a single comment
{%
set foo = {"1": 2, "3", []}
%}
{% set foo = {
"1": 2,
"3", []} %}



# SYNTAX 3./1.1
# keys, values and sections outside of the templating code should display
# as normal
a = {{ b }}
{{ a }} = b
a = {{ b + 1 }}
a = {{ b_b | c.d('1', "2", 3) }}
[[{{ a }}]]
d = e # check highlighting of this line un-affected by section
[[{{ a_a | c.d('1', "2", 3) }}]]
d = e # check highlighting of this line un-affected by section
[[a{{ b }}c]]
d = e # check highlighting of this line un-affected by section
[[{{ "%(foo)s" % {"foo": "abc"}}}]]
oliver-sanders marked this conversation as resolved.
Show resolved Hide resolved
d = e # check highlighting of this line un-affected by section

# SYNTAX 3./1.2
# comments outside of the templating code should display as normal
{# foo #} # foo
{% foo %} # foo
{{ foo }} # foo

# SYNTAX 3./1.3
# tasks and deps outside of the templating code should display as normal
graph = """
foo => {{ bar }} => qux
foo => {{ b_ar | baz.pub('1', "2", 3) }} => qux =>
{% if BAR %}
bar => pub
{% else %}
baz => pub
{% endif %}
"""