pg_query.proto: Add Token ASCII_36; Add scan test

[evanj]

Postgres's scanner will scan $ into a single token. For example:

    SELECT $identifier;

The parser will later reject this. However, since the scanner returns this value, we should define a Token for it in the protobuf.

test/scan.c:

• Add a check so the test does not crash if token_kind is NULL, which it was before I defined the token. The test now prints an error:

    INVALID result for "...": scan_result token 1 token_kind == NULL

• Make tests constant to prevent accidental modification. Unnecessary but could be helpful.
• Define testsCount automatically so it no longer needs to be changed.
• Add an assert to help ensure tests are defined correctly.

[lfittl]

Looks good, and thanks for adding tests!

FWIW, whilst it doesn't really make sense to have queries with non-numeric parameter identifiers, it doesn't hurt to support this case since its a valid output of the scan function (which as you note then gets rejected by the parser in regular Postgres).

Out of curiosity, is there a particular use case where you have queries written like this?

[evanj]

As far as I know so far: this $ token is always a parse error. According to my reading of the documentation, Postgres does not permit $ as an operator, but it does permit it:

• In parameter placeholders: SELECT $5;
• After the first character of an identifier: CREATE TABLE valid$identifier ();
• As dollar quote: SELECT $$string with $ lots of stuff$$;
• As a "named" dollar quote: SELECT $name$my string$name$;

I found this because I wrote a fuzz test for a tool that is rewriting Postgres queries using https://github.com/pganalyze/pg_query_go , and it happened to generate this. The "unsupported" token caused my fuzz test to be unhappy.

So: This isn't "needed", except for completeness for supporting any Postgres tool.