[RFC] Numbers #1877
Labels
Comments
|
I think we are going to move forward with: I'm not convinced the fourth-level scopes are worth standardizing, but I guess most of the syntaxes are using them in some form. |
|
This will be part of the docs in the next release |
deathaxe
pushed a commit
to deathaxe/sublime-packages
that referenced
this issue
Feb 16, 2019
This commit applies numeric constant scope names with regards to the updated guideline which is a result of a proposal at sublimehq#1877.
wbond
pushed a commit
that referenced
this issue
Oct 16, 2019
* [Erlang] Rewrite Syntax Fixes #1609 Fixes #1724 According to #481 this PR provides an Oniguruma free Erlang syntax definition written from scratch. Outline ======= The syntax definition is based on the http://erlang.org/doc and includes all sematic features of Erlang/OTP 21.2 which are required for highlighting. It follows https://www.sublimetext.com/docs/3/syntax.html and includes concepts from https://github.com/sublimehq/Packages. Nearly every aspect and detail of the syntax definition was designed from ground up making use of the latest features of ST's syntax engine and with the intent to apply the most recent best practice examples found in other recently updated syntax definitions. Changes ======= * Add support for Erlang ShellScript (escript) * Add support for Erlang Type Language (`-type`, `-spec` directive) * Add lots of detailed test cases * Add some invalid illegal matching where appropriate * Add builtin constants and macros highlighting * Improve the list of builtin functions ([BIFs](http://erlang.org/doc/man/erlang.html#exports)) * Improve indention settings * Improve shebang/editorconfig detection in `first_line_match` * Improve support for _Goto Definition_ and _Goto Reference_ * Improve parsing performance by round about 50% * Fix quoted atom highlighting (function names, module names, field names, ...) * Fix bitstring data type highlighting * Fix string escape highlighting * Fix string placeholders highlighting * Fix identifier break handling (`@` is no word-break) * Fix lots of context pop offs * ... Benchmarks ========== The whole design and development process included benchmarking of different solutions to find the best possible performance for the different sets of rules. The following examples provide an impression of the performance improvements achieved with the new design. **10k binary** Bin = << << (X*2) >> || <<X>> <= << 1,2,3 >> >>. **10k maps** Expr#{name=>"adam",{age,24}:=fct(),4.0=>{july,29}} **10k records** #record{field1="val1", Field2=3, field3, 'Field-4'={}, _=atom} **10k fun types** -type fun() :: fun(() -> int()). **10k spec** -spec is_foo(Pid) -> boolean() when Pid :: pid() | if(); (Pid) -> ok. **ErlangOTP files** https://github.com/erlang/otp/blob/master/lib/wx/include/gl.hrl https://github.com/erlang/otp/blob/master/lib/stdlib/test/re_testoutput1_split_test.erl https://github.com/erlang/otp/blob/master/lib/xmerl/test/xmerl_sax_std_SUITE.erl https://github.com/erlang/otp/blob/master/lib/crypto/test/crypto_SUITE.erl Results ======= Type | before | after | difference -------------------------------|------------|-------------|------------ 10k binaries | 303ms | 140ms | -53.8% 10k maps | 326ms | 156ms | -52.1% 10k records | 399ms | 130ms | -67.4% 10k fun type | n.a. | 79ms | 10k spec | n.a. | 176ms | gl.hrl | 47ms | 24ms | -48.9% re_testoutput1_split_test.erl | 448ms | 305ms | -31.9% xmerl_sax_std_SUITE.erl | 389ms | 127ms | -67.4% crypto_SUITE.erl | 79ms | 27ms | -65.8% * [Erlang] Apply numeric scope guideline This commit applies numeric constant scope names with regards to the updated guideline which is a result of a proposal at #1877. * [Erlang] Fix anonymous function definitions This commit applies suggestions from the review process. Including anonymous function definitions into goto definition features makes not much sense on the one hand and brings too much noise on the other. The anonymous function maybe assigned to a variable, but this is a dynamic reference to the function then, which can be redefined at any point. Hence scoping the l-value of an anonymous function definition as `entity.name.function` is removed. It's scoped as ordinary `variable.other` instead. * [Erlang] Fix record scopes This commit applies suggestions from the review process. A record is defined only by `-define(record_name, {...})` preprocessor. Everything else can be considered usage. This simplifies the `record` context a little bit. * [Erlang] Remove anonymous functions from goto definition * [Erlang] Add missing MACRO to symbol list This commit adds macro definitions to the symbol list. * [Erlang] Fix record field scopes As the local creation/copy of records is no longer scoped as definition the logical result is to handle the field access within the creation expression as normal field-write-access instead of a field definition as well. To keep the field definition within preprocessor expressions intact, the `record-body` is now assigned to record definitions only. A copy and modified `record-fields` context is used within record creation expressions. Example: Expr#rec{field=1} ^^^^^ variable.other.field Note: We might probably think about more general scopes for fields and records? Maybe: `variable.other.field` -> `variable.other.member` `variable.other.record` -> `variable.other.struct` or `variable.qualifier`?? * [Erlang] Fix spec/type scopes This commit fixes an issue with clearing too many scopes in the -spec() and -type() preprocessor expressions, which caused the main scope `source.erlang` to be removed by accident. * [Erlang] Remove obsolete symbol list override As anonymous function definitions were removed recently, the corresponding and therefore obsolete symbol list definition can be removed as well. * [Erlang] Update indexed symbol and reference lists This commit... 1. removes `entity.name.record` from the references this scope was recently removed from all "usage" use-cases. 2. cleans the indexed symbol list a bit. * [Erlang] Add meta.atom.erlang This commit adds the `meta.atom.erlang` scope to all atoms. Note: It's handled optional and therefore no tests are added. * [Erlang] Fix fun scopes This commit fixes an issue with clearing too many scopes in `fun()` expressions. * [Erlang] Update type test cases This commit extends the test cases of `-type` expressions to verify not to clear too many scopes in any situation. * [Erlang] Fix macro in indexed symbol list * [Erlang] Tweak symbol list This commit applies review suggestions to the symbol list. 1. As all definitions are made in the preprocessor statements, the corresponding keywords are used as prefix for the symbol list item labels. < -> -export > -> -import # -> -record :: -> -spec, -type Ξ -> removed ... 2. Merges the "Function Macro" list into "Macro". 3. Fixes an issue with exports not being listed. 4. Fixes an issue with function definitions not being listed, which use macros as name (e.g.: `?macro(args) -> ok.`). * [Erlang] Fix some emergency bailouts This commit adds (some) missing emergency bailouts to pop off from scopes not after a clause's end. Effects: - function parameters - function call arguments - type preprocessor statements - illegal parens/brackets/braces * [Erlang] Fix more emergency bailouts This commit adds (some) missing emergency bailouts to pop off from scopes not after a clause's end. Effects: - groups - lists - maps - record fields - tuples It also adds the `illegal-stray` context to scope the last closing parentheses/bracket/brace `invalid.illegal` in order to indicate an syntax error before the end of the clause. The idea behind is any nested unclosed context to with the `.`. * [Erlang] Add bailout from hex numbers This commit adds a bailout to pop off from a hexadecimal number context not after a clause's end. * [Erlang] Add bailout from binary strings This commit adds a bailout to pop off from a binary string context not after a clause's end. * [Erlang] Remove repetitions This commit creates an `clause-end-or-stray` context to avoid including the two `clause-end-pop` and `illegal-stray` one after each other repetitively. * [Erlang] Add bailout from function parameters This commit makes sure to pop off from parameter lists of function/spec/type definitions even with unbalanced parentheses/... . * [Erlang] Add ... in list typespecs The `...` is used in Erlang's typespec language to indicate an arbitrary number of list items, parameters or tuple items. This commit makes sure not to interfere with emergency bailouts. The variable is used in ErlangOTP\erts\preloaded\src\erlang.erl * [Erlang] Remove meta.function from macro definitions Even though macro definitions may look like function definitions, they are not related with them in any manner. The parameter list is just used to create multi-purpose macros. This commit therefore removes the `meta.function` scopes from expressions: -define(MACRO_NAME(ARG1,ARG2) , ...). * [Erlang] Rename _ scope The `_` variable is called "anonymous variable" by Erlang. Hence the scope is renamed accordingly. * [Erlang] Add variable-anonymous-pop context This commit is to reduce duplicated matches. * [Erlang] Add variable-other-pop context This commit is to reduce duplicated matches. * [Erlang] Reorganize function-call context This commit splits the function-call context into smaller pieces and adds a `function-call-pop` for later use. * [Erlang] Reorganize type-call context This commit splits the type-call context into smaller pieces and adds a `type-call-pop` for later use. The identifier context is moved to the other "atom" contexts. * [Erlang] Reorganize type-fun context This commit splits the type-fun context into smaller pieces and adds a `type-fun-pop` for later use. * [Erlang] Merge fun-type into type-call context Basically the fun-type context contains a special type of type-call. Hence those contexts are merged. This does not change the behavior in any way. * [Erlang] Add meta.path This commit applies the scope naming guideline which says: Complete identifiers, including namespace names, should use the following scope. Such identifiers are the fully-qualified forms of variable, function and class names. * [Erlang] Fix fun call identifier The `variable.function` scope is generally used to scope an function identifier in all default syntaxes. A normal function identifier in Erlang is an atom. Means a normal function name must start with lower case or underscore. my_func_name() On the other hand, Erlang supports the definition of closures via fun keyword, which binds a "function" to a real variable. // define the function MyLocalFunc = fun() ... end, // call the funciton MyLocalFunc(), Scoping both, an ordinary function identifier and a closure as `variable.function` is therefore confusing. As we already support constants or macros to be used as identifiers %MY_FUNC_MACRO() it is logical to scope `MyLocalFunc` as `variable.other` because it is a real variable, which holds a function. * [Erlang] Fix fun definition identifier This commit applies the philosophy of the previous commit to `fun` definitions and renames the `entity.name.function` to `variable.other` for local fun identifiers. Fun1 = fun FunName(X) -> X+ FunName(X) end. ^ rename entity -> variable * [Erlang] Fix fun definition identifier part 2 A local fun identifier really needs to be a variable. Invalid: `fun funcName() -> ... end.` Valid: `fun FuncName() -> ... end.` * [Erlang] Fix record field access in macros * [Erlang] Fix mapping separator This commit fixes an issue reported during the review process, which caused the `:=` key-value separator not to be scoped correctly.
mitranim
pushed a commit
to mitranim/Packages
that referenced
this issue
Mar 25, 2022
* [Erlang] Rewrite Syntax Fixes sublimehq#1609 Fixes sublimehq#1724 According to sublimehq#481 this PR provides an Oniguruma free Erlang syntax definition written from scratch. Outline ======= The syntax definition is based on the http://erlang.org/doc and includes all sematic features of Erlang/OTP 21.2 which are required for highlighting. It follows https://www.sublimetext.com/docs/3/syntax.html and includes concepts from https://github.com/sublimehq/Packages. Nearly every aspect and detail of the syntax definition was designed from ground up making use of the latest features of ST's syntax engine and with the intent to apply the most recent best practice examples found in other recently updated syntax definitions. Changes ======= * Add support for Erlang ShellScript (escript) * Add support for Erlang Type Language (`-type`, `-spec` directive) * Add lots of detailed test cases * Add some invalid illegal matching where appropriate * Add builtin constants and macros highlighting * Improve the list of builtin functions ([BIFs](http://erlang.org/doc/man/erlang.html#exports)) * Improve indention settings * Improve shebang/editorconfig detection in `first_line_match` * Improve support for _Goto Definition_ and _Goto Reference_ * Improve parsing performance by round about 50% * Fix quoted atom highlighting (function names, module names, field names, ...) * Fix bitstring data type highlighting * Fix string escape highlighting * Fix string placeholders highlighting * Fix identifier break handling (`@` is no word-break) * Fix lots of context pop offs * ... Benchmarks ========== The whole design and development process included benchmarking of different solutions to find the best possible performance for the different sets of rules. The following examples provide an impression of the performance improvements achieved with the new design. **10k binary** Bin = << << (X*2) >> || <<X>> <= << 1,2,3 >> >>. **10k maps** Expr#{name=>"adam",{age,24}:=fct(),4.0=>{july,29}} **10k records** #record{field1="val1", Field2=3, field3, 'Field-4'={}, _=atom} **10k fun types** -type fun() :: fun(() -> int()). **10k spec** -spec is_foo(Pid) -> boolean() when Pid :: pid() | if(); (Pid) -> ok. **ErlangOTP files** https://github.com/erlang/otp/blob/master/lib/wx/include/gl.hrl https://github.com/erlang/otp/blob/master/lib/stdlib/test/re_testoutput1_split_test.erl https://github.com/erlang/otp/blob/master/lib/xmerl/test/xmerl_sax_std_SUITE.erl https://github.com/erlang/otp/blob/master/lib/crypto/test/crypto_SUITE.erl Results ======= Type | before | after | difference -------------------------------|------------|-------------|------------ 10k binaries | 303ms | 140ms | -53.8% 10k maps | 326ms | 156ms | -52.1% 10k records | 399ms | 130ms | -67.4% 10k fun type | n.a. | 79ms | 10k spec | n.a. | 176ms | gl.hrl | 47ms | 24ms | -48.9% re_testoutput1_split_test.erl | 448ms | 305ms | -31.9% xmerl_sax_std_SUITE.erl | 389ms | 127ms | -67.4% crypto_SUITE.erl | 79ms | 27ms | -65.8% * [Erlang] Apply numeric scope guideline This commit applies numeric constant scope names with regards to the updated guideline which is a result of a proposal at sublimehq#1877. * [Erlang] Fix anonymous function definitions This commit applies suggestions from the review process. Including anonymous function definitions into goto definition features makes not much sense on the one hand and brings too much noise on the other. The anonymous function maybe assigned to a variable, but this is a dynamic reference to the function then, which can be redefined at any point. Hence scoping the l-value of an anonymous function definition as `entity.name.function` is removed. It's scoped as ordinary `variable.other` instead. * [Erlang] Fix record scopes This commit applies suggestions from the review process. A record is defined only by `-define(record_name, {...})` preprocessor. Everything else can be considered usage. This simplifies the `record` context a little bit. * [Erlang] Remove anonymous functions from goto definition * [Erlang] Add missing MACRO to symbol list This commit adds macro definitions to the symbol list. * [Erlang] Fix record field scopes As the local creation/copy of records is no longer scoped as definition the logical result is to handle the field access within the creation expression as normal field-write-access instead of a field definition as well. To keep the field definition within preprocessor expressions intact, the `record-body` is now assigned to record definitions only. A copy and modified `record-fields` context is used within record creation expressions. Example: Expr#rec{field=1} ^^^^^ variable.other.field Note: We might probably think about more general scopes for fields and records? Maybe: `variable.other.field` -> `variable.other.member` `variable.other.record` -> `variable.other.struct` or `variable.qualifier`?? * [Erlang] Fix spec/type scopes This commit fixes an issue with clearing too many scopes in the -spec() and -type() preprocessor expressions, which caused the main scope `source.erlang` to be removed by accident. * [Erlang] Remove obsolete symbol list override As anonymous function definitions were removed recently, the corresponding and therefore obsolete symbol list definition can be removed as well. * [Erlang] Update indexed symbol and reference lists This commit... 1. removes `entity.name.record` from the references this scope was recently removed from all "usage" use-cases. 2. cleans the indexed symbol list a bit. * [Erlang] Add meta.atom.erlang This commit adds the `meta.atom.erlang` scope to all atoms. Note: It's handled optional and therefore no tests are added. * [Erlang] Fix fun scopes This commit fixes an issue with clearing too many scopes in `fun()` expressions. * [Erlang] Update type test cases This commit extends the test cases of `-type` expressions to verify not to clear too many scopes in any situation. * [Erlang] Fix macro in indexed symbol list * [Erlang] Tweak symbol list This commit applies review suggestions to the symbol list. 1. As all definitions are made in the preprocessor statements, the corresponding keywords are used as prefix for the symbol list item labels. < -> -export > -> -import # -> -record :: -> -spec, -type Ξ -> removed ... 2. Merges the "Function Macro" list into "Macro". 3. Fixes an issue with exports not being listed. 4. Fixes an issue with function definitions not being listed, which use macros as name (e.g.: `?macro(args) -> ok.`). * [Erlang] Fix some emergency bailouts This commit adds (some) missing emergency bailouts to pop off from scopes not after a clause's end. Effects: - function parameters - function call arguments - type preprocessor statements - illegal parens/brackets/braces * [Erlang] Fix more emergency bailouts This commit adds (some) missing emergency bailouts to pop off from scopes not after a clause's end. Effects: - groups - lists - maps - record fields - tuples It also adds the `illegal-stray` context to scope the last closing parentheses/bracket/brace `invalid.illegal` in order to indicate an syntax error before the end of the clause. The idea behind is any nested unclosed context to with the `.`. * [Erlang] Add bailout from hex numbers This commit adds a bailout to pop off from a hexadecimal number context not after a clause's end. * [Erlang] Add bailout from binary strings This commit adds a bailout to pop off from a binary string context not after a clause's end. * [Erlang] Remove repetitions This commit creates an `clause-end-or-stray` context to avoid including the two `clause-end-pop` and `illegal-stray` one after each other repetitively. * [Erlang] Add bailout from function parameters This commit makes sure to pop off from parameter lists of function/spec/type definitions even with unbalanced parentheses/... . * [Erlang] Add ... in list typespecs The `...` is used in Erlang's typespec language to indicate an arbitrary number of list items, parameters or tuple items. This commit makes sure not to interfere with emergency bailouts. The variable is used in ErlangOTP\erts\preloaded\src\erlang.erl * [Erlang] Remove meta.function from macro definitions Even though macro definitions may look like function definitions, they are not related with them in any manner. The parameter list is just used to create multi-purpose macros. This commit therefore removes the `meta.function` scopes from expressions: -define(MACRO_NAME(ARG1,ARG2) , ...). * [Erlang] Rename _ scope The `_` variable is called "anonymous variable" by Erlang. Hence the scope is renamed accordingly. * [Erlang] Add variable-anonymous-pop context This commit is to reduce duplicated matches. * [Erlang] Add variable-other-pop context This commit is to reduce duplicated matches. * [Erlang] Reorganize function-call context This commit splits the function-call context into smaller pieces and adds a `function-call-pop` for later use. * [Erlang] Reorganize type-call context This commit splits the type-call context into smaller pieces and adds a `type-call-pop` for later use. The identifier context is moved to the other "atom" contexts. * [Erlang] Reorganize type-fun context This commit splits the type-fun context into smaller pieces and adds a `type-fun-pop` for later use. * [Erlang] Merge fun-type into type-call context Basically the fun-type context contains a special type of type-call. Hence those contexts are merged. This does not change the behavior in any way. * [Erlang] Add meta.path This commit applies the scope naming guideline which says: Complete identifiers, including namespace names, should use the following scope. Such identifiers are the fully-qualified forms of variable, function and class names. * [Erlang] Fix fun call identifier The `variable.function` scope is generally used to scope an function identifier in all default syntaxes. A normal function identifier in Erlang is an atom. Means a normal function name must start with lower case or underscore. my_func_name() On the other hand, Erlang supports the definition of closures via fun keyword, which binds a "function" to a real variable. // define the function MyLocalFunc = fun() ... end, // call the funciton MyLocalFunc(), Scoping both, an ordinary function identifier and a closure as `variable.function` is therefore confusing. As we already support constants or macros to be used as identifiers %MY_FUNC_MACRO() it is logical to scope `MyLocalFunc` as `variable.other` because it is a real variable, which holds a function. * [Erlang] Fix fun definition identifier This commit applies the philosophy of the previous commit to `fun` definitions and renames the `entity.name.function` to `variable.other` for local fun identifiers. Fun1 = fun FunName(X) -> X+ FunName(X) end. ^ rename entity -> variable * [Erlang] Fix fun definition identifier part 2 A local fun identifier really needs to be a variable. Invalid: `fun funcName() -> ... end.` Valid: `fun FuncName() -> ... end.` * [Erlang] Fix record field access in macros * [Erlang] Fix mapping separator This commit fixes an issue reported during the review process, which caused the `:=` key-value separator not to be scoped correctly.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current Situation
ST3 Scope Naming Guideline
Numeric literals, including integers, floats, etc. should use one of:
Used Scopes by Syntaxes
The following list contains all kinds of
constant.numericwhich are used in the Default Syntaxes and are related to a specific data-type. Stuff likeconstant.numeric.versionetc. is filtered out.The Issues
Type and Base
Most (older) syntaxes seem to use the 3rd-level scope to distinguish the type (
integer,float,complex) while the 4th-level scope provides information about the base (base-<n>,generic-basebinary,decimal,hexadecimal,octal).The official ST documentaion instead mixes
typeandbasein the 3rd-level scope and is applied to few recently reworked syntaxes only.This leads to conflicts like
constant.numeric.decimalvs.constant.numeric.integer. Both scopes describe an integer, butdecimalmeans base 10 while integer can be anything. Same conflict exists forhexandbinary.Naming
While most (older) syntaxes use
hexadecimalthe official guideline sayshex. Butbinaryis not shortent tobin.Proposal
All scopes should use consistant short or long names.
binorbinarydecordecimalhexorhexadecimalintorintegeroctoroctal...
Distinguish data type and base system.
The majority of syntax definitions uses
constant.numeric.<type>.<base>for a good reason. R even distinguishes the base forfloat. We should keep this naming scheme avoid mixing up data types and base systems.Alternative
Based on the existing ST scope naming guideline, the following scheme can be created, which implicitely defines
bin,oct,decandhexas integers.The
realandimaginarypart ofcomplexnumbers is distinguished but may implicitely be assumedfloat.All integers with other base systems are grouped in
other.The text was updated successfully, but these errors were encountered: