[RFC] More specific subscopes (e.g. meta.function.name)

[Thom1729]

From the scope naming guidelines:

The entire scope of a function should be covered by one of the following scopes. Each variant should be applied to a specific part, and not stacked. For example, meta.function.php meta.function.parameters.php should never occur, but instead the scopes should alternate between meta.function.php then meta.function.parameters.php and back to meta.function.php.

• meta.function
• meta.function.parameters
• meta.function.return-type

This is an excellent system, and I think that it can be extended gracefully to cover other parts of a function. The goal would be to provide standard subscope names so that syntaxes that exceed the minimum specificity requirements are compatible with each other.

An important point raised by wbond is that "the more specific it gets, the more difficult to standardize due to language semantics and differences." Any additions should be conservative; there's little point in standardizing a construct that only appears in a single common language. However, universality is not required; for instance, many languages will never use meta.function.return-type, but enough do that it is well worth having a standard for it.

The following sorts of constructs seem like good candidates for a standardized subscope:

• meta.function.name. In many languages, this may always be marked with entity.name.function, which would seem to make this scope redundant. But this is not always true. Take JavaScript:

  class Foo {
      myFunc() {}
      2e5() {} // The name of this function is 200000
      'another function'() {}
      [getFunctionName()]() {}
  }

  Or SQL:

  create function "foo" ...

• meta.function.body. In some languages, such as C, this is delimited by punctuation like braces. In other languages, such as Python, it is not delimited by punctuation at all. In yet other languages, such as JavaScript, it varies:

  (function() { return 42; });
  (() => { return 42; });
  (() => 42);

• meta.function.access. In many languages, this may be more than one word, such as C#:

  class Foo {
      protected internal void Bar() {}
  }

• meta.function.type-parameters. Many languages explicitly list type or template parameters, such as Java:

  class Foo {
      void Bar<T>() {}
  }

Other subscopes are conceivable, but I think that these four (along with meta.function.parameters and meta.function.return-type) cover the pertinent parts of function declarations and expressions while still being general enough to be used in a wide variety of common languages. In particular, meta.function.name and meta.function.body could pertain to every language that has functions.

Standardizing these scopes would not (and should not) force all syntax authors to use them, but it would help authors who want to provide a full-featured syntax (or who rely on more specific scopes for features like autocompletion) to create syntaxes that are consistent and interoperable. If these scopes did see wide adoption, then they would be useful hooks for color scheme authors and even (perhaps) for simplistic code-analysis tools.

For the above reasons, I would like to discuss adding additional subscopes such as the ones suggested here to the official scope naming guidelines. I'd be interested in feedback both for the idea of subscopes in general and for the scopes suggested above.

[deathaxe]

I would welcome those parts as standards and already define meta.function.name in one of my syntax packages because of the mentioned reason.

I also define a meta.function.attributes scope. So I am wondering if it could be used as a kind of more general variant of meta.function.access, too. Many object orientated languages provide access restrictions to member so it is obvious to define a scope for, but could something like meta.function.attributes be used for it, too?

[Thom1729]

That could be useful.

I'd lean toward keeping attributes and access separate, because several common languages have both and use different syntax for them. Python is an exception, because many features that most languages implement as special syntax are instead implemented instead as decorators.

Another concern is that in many languages that use function decorators it may not be generally possible to know ahead of time that they are part of a function declaration, because the decorators are conventionally followed by newlines. For example, in both Python and JavaScript (proposed), a decorator could precede either a function or a class, and there's no way to know which. C# attributes are even more permissive.

Are there common languages that do not suffer this difficulty? I saw your mention of G-code, but I'm not familiar with it.

[deathaxe]

Decorators and things like that are already discussed in #737. The results are already used by recently updated syntax files like Scala. So normal function scopes don't need to take this into account anymore. The meta sub-scopes you propose are very fine, but I just missed that general attribute thing.

Yes I thought about s84d_gcode syntax. The SINUMERIK CNC allows function definitions like

PROC name(INT arg1, REAL arg2) DISPLOF SBLOF

where DISPLOF and SBLOF are attributes or properties to define how the function is handled. DISPLOF tells the interpreter not to display any statement of this function while execution. This is something language specific and thus shouldn't be too interesting here.

They just don't match any of your proposed scopes and thus the idea raised to use meta.function.attribute instead of meta.function.access as this is quite special.

Take a look on the storage scope. It defines

• storage.type
• storage.modifier

with storage.modifier being used for public, protected, const, etc.

Following this rule, we would need to define a minimum required function.modifier scope in general as parent for all kind of additional information.

• function.modifier
• function.modifier.access
• function.modifier.attributes
• ...

[Thom1729]

Are there other languages that would use the ...attributes scope? I'm having a hard time mapping those attributes from G-code onto other languages.

[deathaxe]

Not sure, maybe most other languages most commonly use the annotation or decorator scope for such things.

[wbond]

We've standardized that decorators, attributes and annotations will use the scopes in #709.

[deathaxe]

@wbond: The point is, that #709 is all about annotations or decorators which are located on a separate line above the function definition. I understood them as kind of pragma to modify functions on compile time.

Consider the meta.function.access from first post being understood as attribute and thus handled as standardized in #709. This would mean to start the line with meta.annotation followed meta.function scope. What would happen, if the meta.function.access was somewhere in the middle of the function definition? Would we break the meta.function scope then?

   void private func()
   ^^^ meta.function.return-type
        ^^^^^^^ meta.annotation.variable
                ^^^^ meta.function.name
                    ^^ meta.function.parameters

I think everything within the function definition line should start with meta.function.

[wbond]

@deathaxe Well, private should be storage.modifier.

In terms of annotations the meta.function, there is not reason they can't be layered. Annotations can be associated with function or all sort of different constructs, so there it would result is scope divergence if we tried to specialize meta.function for languages that have pre-defined places where annotations can exist.

If you check out the existing C# syntax and tests, you'll see we have annotation before methods and in the middle of parameters. There is no harm in having meta.function meta.annotation variable.annotation. Stacking scopes makes sense when the constructs can exist in multiple places. Specializing makes sense where stacking results in duplication. Having meta.function meta.function.return-type is silly when we can just specialize the return type part.

Does that make sense?

[deathaxe]

Yes this makes sense, if annotations aren't handled as part of functions at all. I maybe was misled, but have a look on python's implementation of annotation as part of the function scope. This took me to the decision to use the meta.function.attribute scope name in the end.

def start_timer(delay: int) -> None:
#                      ^^^ meta.function.parameters.annotation
#                              ^^^^^ meta.function.annotation.return.python

A look into the latest C# syntax makes the usage of annotations completely clear to me, but I found something I dislike with the meta.function or meta.method handling there. Maybe it's worth an own issue, but the meta.function scope should not start with the function name, but include all parts of the function declaration line.

Current situation:

class YourMainClass
{
  public void Run([Usage("help text")] int x, int y);
//            ^^^ meta.method
//               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.method.parameters
}

At least meta.method should start with the first keyword.

class YourMainClass
{
  public void Run([Usage("help text")] int x, int y);
//^^^^^^^^^^^^^^^ meta.method
//               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.method.parameters
}

When following post 1 with .modifier. as general scope included into .access it should look like:

class YourMainClass
{
  public void Run([Usage("help text")] int x, int y);
//^^^^^^^ meta.method.modifier.access
//       ^^^^ meta.method.return-type
//           ^^^^ meta.method.name
//               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ meta.method.parameters
}

The usage of modifier would not limit the meta-scope to access rights but would allow any kind of modifiers in that place.

Can it be considered standardized to use meta.method in place of meta.function in classes?

[wbond]

The reason it doesn't start at the beginning of the line is because the language is ambiguous until an identifier followed by a (, =, or ; is found. Since some users like to format methods/functions onto multiple lines, and the syntax highlighting engine only processes one line at a time, we sometimes have to be conservative. Lookaheads can be used in some situations.

When it comes down to it, properly lexing C-like languages is a real pain, and will never be 100% because of:

• C preprocessor
• No leading function/method keyword (e.g. def in Python) and multiline identifiers/declarations

[wbond]

In terms of Python, yes we may want to revisit that. I believe that was written before we standardized on scopes. I haven't looked into it enough to see if the annotations scopes are applicable there, or if they should be scoped as something else.

[deathaxe]

Thanks for clarification.

properly lexing C-like languages is a real pain

This is the drawback of the possibility of context sensitive lexing. The rules basically need to imitate the behavior of the compiler/interpreter to correctly handle all the grammar required especially for meta scopes.

The meta scoping shouldn't waste too much efforts as meta's are not primarily seen by user but "only" may support package devs to provide proper auto-completions and such things.

Means it's not a big deal not to start meta.function at the first keyword. :-)

[Thom1729]

meta.method makes sense, but I'm not sure that it adds substantial expressiveness. Won't a function definition at the top level of a class definition be a method in virtually every language? In addition, there are languages like Ruby and JavaScript where the distinction between a function and a method is blurry to begin with.

I think that if meta.declaration ever gets standardized, then it might at that point be worth distinguishing a function declaration from a method declaration.

[FichteFoll]

I second @Thom1729's concern. Java, for example and as far as I'm aware, doesn't even have "functions" at all, since it only allows "function" definitions inside classes and these are then called "static methods".

Alas, methods are just functions that operate on a specific object that is passed to the function in one way ore another, so imo the notion of a "method" does not provide any meaningful information over "function" (for scope names).

In Pascal, you also have "procedures" that are just functions without a return value (which makes sense semantically, but in the real world the term "function" has just established itself as the standard name).

Regarding Python function annotation scopes: Those most likely still need to be revised. I've only looked into decorators.

[deathaxe]

As the goal is to find common scope names which can be reused by all syntaxes I vote for using meta.function for everything which is callable, no matter what a certain language calls it like.

So everything like procedures, functions, methods should use meta.function for declaration and meta.function-call when being used in the code.

[rwols]

As the goal is to find common scope names which can be reused by all syntaxes I vote for using meta.function for everything which is callable, no matter what a certain language calls it like.

I agree 100% with this. The line between functions and methods is blurry anyway.