[BUG] Compilation error after execution of updateKeywords

[ssteinhauser]

The execution of the updateKeywords task adds the following tokens to RelObjectNameWithoutValue:

JSqlParser/src/main/jjtree/net/sf/jsqlparser/parser/JSqlParserCC.jjt

Lines 529 to 551 in 411a3da

	| <#TYPE_BIT: "BISTRING">
	| <#TYPE_BLOB: "BLOB" | "BYTEA" | <K_BINARY> | "VARBINARY" | <K_BYTES> >
	| <#TYPE_BOOLEAN: "BOOLEAN" | "BOOL" >
	| <#TYPE_ENUM: "ENUM" >
	| <#TYPE_MAP: "MAP" >
	| <#TYPE_DECIMAL: "DECIMAL" | "NUMBER" | "NUMERIC" >
	| <#TYPE_TINYINT: "TINYINT" | "INT1" >
	| <#TYPE_SMALLINT: "SMALLINT" | "INT2" | "SHORT" >
	| <#TYPE_INTEGER: ( "INTEGER" | "INT" | "INT4" | <K_SIGNED> | <K_UNSIGNED> ) >
	| <#TYPE_BIGINT: "BIGINT" | "INT8" | "LONG" >
	| <#TYPE_HUGEINT: "HUGEINT" >
	| <#TYPE_UTINYINT: "UTINYINT" >
	| <#TYPE_USMALLINT: "USMALLINT" >
	| <#TYPE_UINTEGER: "UINTEGER" >
	| <#TYPE_UBIGINT: "UBIGINT" >
	| <#TYPE_UHUGEINT: "UHUGEINT" >
	| <#TYPE_REAL: "REAL" | "FLOAT4" | "FLOAT">
	| <#TYPE_DOUBLE: "DOUBLE" | "PRECISION" | "FLOAT8" | "FLOAT64">
	| <#TYPE_VARCHAR: "NVARCHAR" | "VARCHAR" | "NCHAR" | <K_CHAR> | "BPCHAR" | "STRING" | "TEXT" | <K_CHARACTER> | "VARYING">
	| <#TYPE_TIME: "TIMETZ" >
	| <#TYPE_TIMESTAMP: "TIMESTAMP_NS" | "TIMESTAMP_MS" | "TIMESTAMP_S" >
	| <#TYPE_UUID: "UUID">

This causes the javacc-jjtree goal to fail:

Error: Line 1962, Column 313: String token "BISTRING" has been defined as a private regular expression.
Error: Line 1962, Column 1163: String token "ENUM" has been defined as a private regular expression.
Error: Line 1962, Column 1500: String token "HUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 1815: String token "MAP" has been defined as a private regular expression.
Error: Line 1962, Column 3401: String token "TIMETZ" has been defined as a private regular expression.
Error: Line 1962, Column 3527: String token "UBIGINT" has been defined as a private regular expression.
Error: Line 1962, Column 3542: String token "UHUGEINT" has been defined as a private regular expression.
Error: Line 1962, Column 3558: String token "UINTEGER" has been defined as a private regular expression.
Error: Line 1962, Column 3672: String token "USMALLINT" has been defined as a private regular expression.
Error: Line 1962, Column 3689: String token "UTINYINT" has been defined as a private regular expression.
Error: Line 1962, Column 3705: String token "UUID" has been defined as a private regular expression.

[manticore-projects]

Thank you for reporting, I actually broke it myself when introducing complex compound Data Type tokens.
I will definitely fix it, but this will take a couple of days since I am in transit right now. Sorry for any inconvenience.

[ssteinhauser]

Thank you for the quick reply.
I tried to implement and provide a solution. The idea is to parse and iterate over each token definition and don't whitelist anything which is defined within a private token.
Let me know your thoughts on this: #2044

However, there is still an issue, related to #2041. After running the updateKeywords now, PRECISION and BASE64 will be removed. PRECISION is removed, because it is just part of a private token. BASE64 is removed, because numbers in the token values are not being whitelisted due to the condition tokenValue.matches("[A-Za-z]+").