Official list of semantic color names to support

[joshgoebel]

With regard to the idea of using TM copes it'd been mentioned we may need extra UI scopes since TM is mostly covering code highlighting. Things like:

• UI (status bars, etc)
• Terminal stuff (cursor, selection, default text)
• ANSI (allow a theme author to map their colors into ANSI space)

Examples:

term.ansi.0
term.ansi.red
term.ansi.bright-red
ui.cursor.fg
ui.cursor.bg
ui.divider
ui.selection.fg
ui.selection.bg

---

Related: #11

I think the discussion of which semantic scopes to support (and how to name them) is separate from the technical discussion of how to add them to builders so I've created this thread... I suggested our initial pass might just start with the original semantic scopes from the spec.. I'll go ahead and write them out. I already see some gaps though.

This is me just trying to take the official list, copy the names, surmise a little bit of their intent (that class really means "class name"), and then group them better.

Terminals (written in scope form, but this could change)

(gathered from reviwing all terminal templates)

• Alacritty
• Blink
• ConCfg
• Console2
• ConsoleZ
• Crosh
• fish
• fish-shell
• Gnome Terminal
• kermit
• kitty
• Konsole
• MinTTY
• PuTTY
• st
• Termite
• Termux
• Tilix
• Windows Terminal
• XFCE4 Terminal
• Xshell
• Xresources

ui/term.cursor (bg/fg?)
term.dim_color (tilix)
ui/term.highlight (bg/fg?) (tilix)
ui/term.text
ui.selection.background (bg/fg?)
term.url_color (should this be diff from syntax highlight?)
# on many terminals fg/bg can be different from the ANSI palette
ui.foreground
ui.background
foreground/background intense (konsole) ???? (bold???)
reverse cursor (st) - is this just a way of saying bg color?

Questions

• Are text and foreground the same?

Background / Foreground / special

BG_default
FG_default
BG_lighter
BG_selection
invisible
line_highlight
FG_dark / BG_status_bar
FG_light
BG_light
BG_search_text

Markup

markup_link_text
markup_link_url
markup_list
markup_code
markup_italic
markup_bold
markup_quoted

Diffs

diff_inserted
diff_changed
diff_deleted

Source

name_class
name_class_inherited
name_function
name_method
name_support
name_variable
xml_tag
xml_attribute
integer
boolean
constant
string
regex (regular_expression)
escape_chars
attribute_id
heading
keyword
storage
selector
deprecated
embed_tag
caret
delimiters 
operator
comments

From the official list

Originally posted by @joshgoebel in #11 (comment)

[joshgoebel]

It'd be useful to get the major editor template maintainers involved in this discussion I think. I for one like the TextMate scope hierarchy used in many editors (though we can flatten)... the Sublime docs are pretty nice here: https://www.sublimetext.com/docs/scope_naming.html

We already brush up against that with a few of our items, like constant, integer, boolean... With TextMate scoping all of these are constant:

• integer -> constant.numeric.integer
• boolean -> constant.language (since booleans are usually built into languages)

So constant isn't opposed to integer and boolean rather integer and boolean are sub-types of constants/literals... so I think some simple review of these things might help before we settle on final scopes.

[joshgoebel]

I'm thinking for first pass we simplify as much as we can, remove tiers... so I'd lose integer, boolean and constant and replace them with literal... and same splitting hairs over WHAT kind of literal for a future day.

And do we need both name_function and name_method? I say need name_function, which can be applied to functions and methods name.

I think keyword and storage (keyword_storage) are still nice, I wonder if someone who ONLY defines keyword is going to start expecting that to "inherit" to storage... this isn't super hard to handle but I think I'd like to try and avoid such traps if we can.

[joshgoebel]

https://www.sublimetext.com/docs/scope_naming.html

I'm leaning towards just saying we support TextMate scoping names and mapping over the 20 or so that explicitly correspond to an baseXX and then let people go wild with what's left... if someone really wants to declare constant.numeric.integer.binary is red, why should we care?

The advantage being a lot of template designers for editors are already going to be familiar with these... and they've thought of everything (and how it's categorized, etc) so we don't have to repeat any of that work... we can provide some minimal docs and then point people to super helpful results like the sublime scope naming guide, etc...

This would instantly make us one of the best, most complete tools for writing simple text-based schemes for editors. And by holding nothing back I think it'll be easy to find people enthusiastic to get the templates working fully.

There are several guides about which scopes are "more critical" for theme and we could publish our own list for those who just wanted to check all the boxes and then stop...

[forrestli74]

Hi, author of base9-vscode here. I learned a lot about what semantic color is needed while implementing it. I'm obviously biased, but here are my two cents:

1. Tm scopes are not enough to capture the theme. Example: status bar, button color, etc.
2. It's easier to categorize semantic colors into two groups: code highlighting and UI element.
   They are not strict and have some overlap. Example of overlap: separators.
3. This is more debatable, but a good theme design need a primary color for most of UI elements, maybe and a secondary. This also aligns with how material design works.
4. To have a good design, different shades colors are almost required. This also aligns with material design. Mostly different shades of white, primary color. Examples: hover button, disabled button, Highlight selection of options, borders between UI section that should not standout. Different shades of other colors are also needed for highlighting: search results, definition of hovered variables, diffed lines.
5. Different apps may use different colors for the same semantic, primarily due to terminal apps are more limited than full feature GUI, such as vscode. Sometimes colored border with a dark background is more desirable. But terminal apps do not support borders, so a brighter background is used to differentiate sections of the UI. Another example is that for borders that are 1 pixel wide of #ffffff might have the same visual intensity as a border of 1 char wide of #333333. Yet another example is that less featured syntax highlighting engine may highlight things differently, such as highlight.js, vim, notepad2, etc.
6. More importantly, t is easier to develop a build system along with developing a full featured theme, such as vscode.

A point against tm theme is that afaik, vim does not support it, and vim ispretty popular.

After those learnings I have landed on the base9 style guide written here:
https://base9-theme.github.io/guide/template

And semantic variables here: https://github.com/base9-theme/base9-core/blob/main/src/semantic.json

Feel free to take any ideas from my learnings.

EDIT: added a few points and removed follow up comments.

[davidscotson]

Vim/Neovim/Treesitter use a very similar system to the tmThemes for the syntax types,

nvim-treesitter/nvim-treesitter@e2b2d23

That commits shows some of the mappings between treesitter through nvim-treesitter to nvim names, which are basically the same as Vim names:

https://neovim.io/doc/user/syntax.html

So a simple mapping should be possible.

Generally they seem to follow the concept of defining a top level one that inherits down, unless overriden with a more specific color, which means you can get a quick theme going with about 11 color choices, but you can just keep going if you're obsessive, and users have a way to tweak that one little thing they don't like.

For terminal apps, other than the base16 which have different names in each, they seem to also let you specify, background, foreground, cursor foreground/background, selection foreground/background which the semantic.json link above has too.

Diff/markup colors is another one where you can probably inherit by default but it would be good to be able to specify an override.

Vim and other editors don't stop there and continue with things like:

http://vimdoc.sourceforge.net/htmldoc/options.html#'highlight'

[joshgoebel]

Tm scopes are not enough to capture the theme. Example: status bar, button color, etc.

Good point, we'd likely create a custom ui scope for this... on that point we'd be forging new ground, but for all the syntax highlighting stuff we'd already have all the problems solved.

They are not strict and have some overlap. Example of overlap: separators.

Separators? If something existing in both we'd just pick one, and that would be the name that won... one could make two names "the same" but I think that's a bit weird.

Mostly different shades of white, primary color.

Right now Base16/17 provides a gradient ramp (base00-base07) used for UI things (bg/fg/status bar/selection/etc)... I'm not sure adding dynamic colors (shades, fades or variants of palette colors) is in scope for the first revision of Base17 but I'm definitely open to such things in the future.

More importantly, t is easier to develop a build system along with developing a full featured theme, such as vscode.

I don't know what this means.

A point against tm theme is that afaik, vim does not support it, and vim ispretty popular.

No one is suggesting we use tmTheme, only that we adopt the names of the scopes... does it translate perfectly to every editor? No, but if you're familiar with both systems (or look at ref matrial) it'd be pretty easy to map TM scopes back into Vim.

[joshgoebel]

For terminal apps, ... background, foreground, cursor foreground/background, selection foreground/background

Yes, we'd likely need a few custom scopes for terminals... one could probably look to a project like https://github.com/mbadolato/iTerm2-Color-Schemes and make a list of needed scopes... this would probably fall under the ui scope if we go the route of TM named scopes or perhaps a new term scope...

[joshgoebel]

One problem is "unequal" themes... templates have to be able to expect to know what output colors are available.... so if say we added ANSI scopes like term.ansi.bright_red which do seem useful, we'd need to define a DEFAULT value (one of the 16 palette entries)... this could possibly be done using the fact that Base16 themes tend to be hue locked (despite that not being part of the spec)... but we're on very shaky ground.

IE: A theme can't just say "i have no idea what color a cursor should be" or " i have no idea what color ANSI red is"... if we add these things as semantic, then there must be an answer - whether it's a default or we require it to be manually specified.

I really don't want Base17 to be hue locked because it makes all themes start to look the same. So then you either force users to specify named colors - or we can't have a built-in ANSI mapping.

And every new thing we REQUIRE (past the minimum) makes theme authoring harder... already for exact highlighting results with just the originally named semantic scopes you're talking 16 colors + maybe ~ 20 semantic mappings.

So for every scope we define (cursor, fold, etc) we have to map it back into the base16 spec - and waht color reasonably works... we can't just invite new scopes out of the air - unless we're going to create a requirement that all themes include those scopes - which hasn't been discussed at all.

[joshgoebel]

From another thread: diff_added, diff_added_background

This seems obviously (the pairing) but then you quickly realize there would be all sorts of problems to doing this. What if an app doesn't support backgrounds - or if a template author hasn't considered that possibility... Right now (if you pay attention to the guide) it's hard to get a bad result... if your app was sophisticated you might write something like:

# imaginary config where the format
# highlight = [ fg, bg ]
git.added = [ {{ background-hex }} , {{ diff_added }} ]

So since we're specifying the background as the foreground and vise versa we get the fancy diff added highlighting via background color (instead of foreground)...

Yet a simpler app/template might have only:

# a made up config format where `key: color`
added:  {{ diff_added }}

And this works great... both templates work great for their respective apps... now imagine we added both fg and bg colors for everything... now the problem is someone can't rely on diff_added to make SENSE anymore... take the above example:

added:  {{ diff_added }}

This would now be broken because we didn't include diff_added_background.. we'll have black on black... a broken app... and again if you think "well just always use both" again we can't be certain that every application is going to support that...

I'm open to suggestions here but for now it seems safer to me (with our wide ecosystem of supported apps) that if someone wants to get fancy with fg/bg inversion, they are just going to have to do it manually (are you already would with base16, for example)

[actionless]

Are text and foreground the same?

yes