Feat: add the ability to set meta in sql comments

[tobymao]

No description provided.

[georgesittas]

I can see how this adds value, though I am a little worried about the comments getting attached to unintended nodes & this introducing additional complexity to extract the meta..

For example, if the use case is "switch off the normalization of an identifier", we could have:

>>> sqlglot.parse_one("select a /* sqlglot.meta normalization = off */, b")
(SELECT expressions:
  (COLUMN this:
    (IDENTIFIER this: a, quoted: False), comments: [' sqlglot.meta normalization = off ']),
  (COLUMN this:
    (IDENTIFIER this: b, quoted: False)))

Note that the info is not set at the Identifier level here, but at the Column level, because of how comments are processed & attached in expressions at parse time. This makes it a bit awkward to detect the meta, because in normalize_identifier we'd have to check the parent to see if 1) it's a column and 2) has the meta set.

[barakalon]

What's the intended use of this?

[tobymao]

What's the intended use of this?

it provides an escape hatch for users to annotate their sql in some way. for example if a user doesn’t want something optimized

[barakalon]

Ah, interesting.

I agree with @georgesittas 's comment re: difficult to know exactly what node you're annotating. And SQLGlot's support for comments isn't great in general. But I wonder what nodes you'll actually need to annotate in practice - maybe its only things like Select and Table.

[georgesittas]

But I wonder what nodes you'll actually need to annotate in practice - maybe its only things like Select and Table.

The motivating use case was disabling normalization for certain identifiers that are used to access JSON data, for example field in some_json.field. These identifiers are currently normalized according to the dialect's resolution rules, but JSON values don't necessarily follow these rules and so normalizing them could break the query.