Scope subcommand followed by other tokens #23

victor-gp · 2023-07-18T01:24:35Z

Resolves #3

Limitations

We cannot scope the sub-tokens following subcommands
- e.g. "PORT" in adb's tcip PORT
- because first we need to scope Usage lines across the board
  - we cannot target subcommands only, without impacting usage lines
  - so it would cause a lot of inconsistencies on that front
- but this is a feature I'm willing to add, it makes sense because
  - usage lines are important, we should highlight them
  - they seem easy enough to target?
- => Scope usage lines #24
Quite a few Usage lines have already become scoped with these changes
- but they look consistent enough within the same help message
- and it's a change we plan to propagate everywhere anyways

Pending

scope a stray usage line in chown
scope a stray usage line in npm

Not required

minor issues to be fixed at a later date in jq, dd
peccata minuta we can live with in dpkg, ripgrep

Examples

adb:

usage lines:

random keyword=ARG lines:

Terms as in "defined terms". Mostly for subcommands followed by arguments or other subcommands. Which are defined terms themselves, but still not handled. e.g.: "reconnect device kick connection from device side to ..." In terms of regression delta, census: - adb: ok, it motivated this change - chown: not ideal, examples with chown itself are scoped. but acceptable. - npm: idem. - dpkg: not cool, I should limit term sequences to 3-4 terms Some unhandled cases in adb, chown and npm, - reveal the single_2_space variable does't work - show a need to treat definitions that do \n after the terms

I was debugging a problem that didn't exist 😱

It's like it doesn't properly use all the {{long_space}} cases (newline and single_2_space)

But it's kinda weird because in forward --remove LOCAL it scopes the LOCAL but skips the --remove. I guess this skip happens in the option-def+space context? Anyways, worth running the syntest debugger on. But not today!

For ease of use. Other X is less wrong but a little more unwieldy to write and reason about.

To reflect the new change in KEYWORD_SCOPES' name

But I'm considering leaving the later unscoped, or any other 2nd term after a keyword term. Cause it causes regressions in other, more common commands, where it scopes terms in examples. (As in every other delta in this commit)

And it introduces a hell of a regression in the Usage section of a few other commands. This is breaking for me, we'll probably do the 1st keyword only.

But this introduces quite a bit of regression. See syntax_test_cp

The shelcheck directive is explained here: https://www.shellcheck.net/wiki/SC2148

To avoid inconsistent scoping in the usage/examples section of a bunch of help messages. But keep these scope assignations as comments, for future reference.

Only the "pair" bit. This is all so overfitted, to suit adb's help message only, that it's making me feel icky. But, on the other hand, it's also a good census of all kinds of patterns that we don't currently have a proper way to handle. And if it doesn't add too many unnecessary scopes for other help messages, I don't care much, I guess?

We're handling subcommand sub-tokens in an explicit manner, without abstraction / generalization. At least for the time being.

Now we're scoping command keywords in usage lines so as to make them coherent. Otherwise, some help messages show usage lines with and without the command colorized.

I hoped to catch the tricky chown case with this, but I think I know why it's failing: we do not `set: then-pop" in keyword-seq-def matches !! As a side effect, we scoped "operand=VALUE" operands in dd, which is nice. As another side effect, we scoped "or:" usage lines in vim, which is bad but easy to fix (next commit).

Cause it's an alternative connector in Usage lines

I hoped to catch the chown corner case with this but no such luck. Instead, we caught a stray adb bug that I had forgotten all about (but my syntax tests didn't :D ).

To harmonize with the following line, where "bat" is scoped. This is very hacky but highlighting bat's help message properly is a big priority for this syntax, as bat is our only playground. Also renames the test file, cause it was incorrectly versioned 23.0.0

This change also has lots of side-effects. ghc, ghci and tealdeer are ok It leaves the jq in the "usage: jq ..." line not scoped, but it's acceptable to me. Not like bat's case.

Cause keyword=VALUE looks pretty ugly if only the former is colorized.

I moved scripts/env.sh to ./utils in 57748ff and renamed the regression diff functions from gdiff* to reg* in f8bf10d

Because it was matching the 2-space with the first lookahead, without consuming the characters it matched. Then it matched the second (negative) lookahead with the same 2-space. But because it's negative, the whole match failed. We just have to circumscribe the 2nd lookahead (negative) within the first one. This catches the 2 stray corner cases that I wanted to fix before merging this branch: one line each in chown and npm. But it also adds a hell of a regression in that it now matches plain text from a lot of help messages. But this can be fixed easily enough, cf. next commit.

The pattern we incorrectly matched was lines including some kind of justified text, with ". " We can safely match this pattern out.

Manual update (as opposed to Dependabot's) because the highlighting for the test help message has changed. It's all good because it's as intended, an improvement. See victor-gp/cmd-help-sublime-syntax#23

victor-gp added 30 commits July 18, 2023 00:49

Fix broken adb syntax test

80382b0

I was debugging a problem that didn't exist 😱

Add tests showing what's wrong with other-term-seq

beae871

It's like it doesn't properly use all the {{long_space}} cases (newline and single_2_space)

Scope argument after subcommand

dd43878

But it's kinda weird because in forward --remove LOCAL it scopes the LOCAL but skips the --remove. I guess this skip happens in the option-def+space context? Anyways, worth running the syntest debugger on. But not today!

Update syntax tests for other-def sequences

db357da

Move other-def-seq include to other-def

83941df

Rename other-def -> keyword, other-variable -> variable

4094410

For ease of use. Other X is less wrong but a little more unwieldy to write and reason about.

Rename "..other.def.." scope to "keyword"

1dbc324

To reflect the new change in KEYWORD_SCOPES' name

Annotate a refactor idea for enum1|enum2|enum3 args

adc4446

Scope "subcommand --option"

09e84e7

But I'm considering leaving the later unscoped, or any other 2nd term after a keyword term. Cause it causes regressions in other, more common commands, where it scopes terms in examples. (As in every other delta in this commit)

Scope "keyword [--option]"

32cdd01

Scope "keyword [ARGUMENT]"

437fc88

And it introduces a hell of a regression in the Usage section of a few other commands. This is breaking for me, we'll probably do the 1st keyword only.

Scope "keyword ARGUMENT"

ceefab3

But this introduces quite a bit of regression. See syntax_test_cp

Require a long space after "keyword ARG" to fix regressions

2157efe

Add gdiffm util to get regression diff with main

9887fbd

Rename scripts/env.sh to ../utils

57748ff

The shelcheck directive is explained here: https://www.shellcheck.net/wiki/SC2148

Don't scope the 2nd term after a subcommand

a064b0f

To avoid inconsistent scoping in the usage/examples section of a bunch of help messages. But keep these scope assignations as comments, for future reference.

Handle "disconnect [HOST[:PORT]]" from adb

f59e14e

Handle "shell [-e ESCAPE] ..." from adb

e42fe00

Handle "shell [-e ESCAPE] ..." from adb

11071bb

Handle "reboot [bootloader|recovery|..." from adb

938f74b

Fix a stray, outdated syntax assertion

6a70341

Rename regression test utils

f8bf10d

Reorganize & document dev utils

310a626

Remove subcommand-def-post

bd64ca0

We're handling subcommand sub-tokens in an explicit manner, without abstraction / generalization. At least for the time being.

Handle "bat <SUBCOMMAND>" from bat

3de9f62

Now we're scoping command keywords in usage lines so as to make them coherent. Otherwise, some help messages show usage lines with and without the command colorized.

Don't scope "or:" as keyword other

833cbb9

Cause it's an alternative connector in Usage lines

Properly set: then-pop after each keyword-seq-def match

751858f

I hoped to catch the chown corner case with this but no such luck. Instead, we caught a stray adb bug that I had forgotten all about (but my syntax tests didn't :D ).

victor-gp added 13 commits July 18, 2023 02:14

Scope bat in "Usage: bat"

aa81feb

To harmonize with the following line, where "bat" is scoped. This is very hacky but highlighting bat's help message properly is a big priority for this syntax, as bat is our only playground. Also renames the test file, cause it was incorrectly versioned 23.0.0

Handle "mount [options] ..." from mount

a700040

This change also has lots of side-effects. ghc, ghci and tealdeer are ok It leaves the jq in the "usage: jq ..." line not scoped, but it's acceptable to me. Not like bat's case.

Change a todo comment priority

22ceeb2

Cause keyword=VALUE looks pretty ugly if only the former is colorized.

Add lookahead_long_space variable

7e46345

Rename lookahead_40 to _lte_40_chars

01e0bbf

Isolate keyword-def-seq last resort matches

5950372

docs: Update scripts/{env.sh -> utils}

c862758

I moved scripts/env.sh to ./utils in 57748ff and renamed the regression diff functions from gdiff* to reg* in f8bf10d

docs: Write command line arguments <LIKE_THIS>

682bc7d

docs: Update the Regression Tests section

76dd911

docs: Development: move a subsection

7e18cd8

Don't scope plain words with lookahead_single_2_space

07b0b4e

The pattern we incorrectly matched was lines including some kind of justified text, with ". " We can safely match this pattern out.

Add a todo for adb's lowercase headings

fc97338

victor-gp mentioned this pull request Jul 31, 2023

Scope usage lines #24

Open

victor-gp marked this pull request as ready for review July 31, 2023 01:02

Merge branch 'main' into scope-subcommand-followed-by-args

2800545

victor-gp merged commit 143fffb into main Jul 31, 2023

victor-gp deleted the scope-subcommand-followed-by-args branch July 31, 2023 02:04

victor-gp mentioned this pull request Dec 29, 2023

Bump cmd-help syntax to b150d84 sharkdp/bat#2819

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Scope subcommand followed by other tokens #23

Scope subcommand followed by other tokens #23

victor-gp commented Jul 18, 2023 •

edited

Loading

Scope subcommand followed by other tokens #23

Scope subcommand followed by other tokens #23

Conversation

victor-gp commented Jul 18, 2023 • edited Loading

Limitations

Pending

Not required

Examples

victor-gp commented Jul 18, 2023 •

edited

Loading