Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema support mixed Chinese and English #3440

Merged
merged 4 commits into from
Dec 13, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions src/parser/parser.yy
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ static constexpr size_t kCommentLengthLimit = 256;
%token <boolval> BOOL
%token <intval> INTEGER
%token <doubleval> DOUBLE
%token <strval> STRING VARIABLE LABEL IPV4 CHINESE_LABEL
%token <strval> STRING VARIABLE LABEL IPV4

%type <strval> name_label unreserved_keyword predicate_name
%type <expr> expression
Expand Down Expand Up @@ -406,7 +406,6 @@ static constexpr size_t kCommentLengthLimit = 256;

name_label
: LABEL { $$ = $1; }
| CHINESE_LABEL { $$ = $1; }
| unreserved_keyword { $$ = $1; }
;

Expand Down
18 changes: 13 additions & 5 deletions src/parser/scanner.lex
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ IS_NOT_NULL (IS{blanks}NOT{blanks}NULL)
IS_EMPTY (IS{blanks}EMPTY)
IS_NOT_EMPTY (IS{blanks}NOT{blanks}EMPTY)

LABEL ([a-zA-Z][_a-zA-Z0-9]*)
DEC ([0-9])
EXP ([eE][-+]?[0-9]+)
HEX ([0-9a-fA-F])
Expand All @@ -46,9 +45,18 @@ IP_OCTET ([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])

U [\x80-\xbf]
U2 [\xc2-\xdf]
U3 [\xe0-\xef]
U3 [\xe0-\xee]
U4 [\xf0-\xf4]
CHINESE_LABEL ({U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U})+
CHINESE {U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U}
CN_EN {CHINESE}|[a-zA-Z]
CN_EN_NUM {CHINESE}|[_a-zA-Z0-9]
LABEL {CN_EN}{CN_EN_NUM}*

U3_FULL_WIDTH [\xe0-\xef]
CHINESE_FULL_WIDTH {U2}{U}|{U3_FULL_WIDTH}{U}{U}|{U4}{U}{U}{U}
CN_EN_FULL_WIDTH {CHINESE_FULL_WIDTH}|[a-zA-Z]
CN_EN_NUM_FULL_WIDTH {CHINESE_FULL_WIDTH}|[_a-zA-Z0-9 ]
LABEL_FULL_WIDTH {CN_EN_FULL_WIDTH}{CN_EN_NUM_FULL_WIDTH}*

%%

Expand Down Expand Up @@ -479,7 +487,7 @@ CHINESE_LABEL ({U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U})+
// Must match /* */
throw GraphParser::syntax_error(*yylloc, "unterminated comment");
}
\`{CHINESE_LABEL}\` {
yixinglu marked this conversation as resolved.
Show resolved Hide resolved
\`{LABEL_FULL_WIDTH}\` {
yylval->strval = new std::string(yytext + 1, yyleng - 2);
if (yylval->strval->size() > MAX_STRING) {
auto error = "Out of range of the LABEL length, "
Expand All @@ -488,7 +496,7 @@ CHINESE_LABEL ({U2}{U}|{U3}{U}{U}|{U4}{U}{U}{U})+
delete yylval->strval;
throw GraphParser::syntax_error(*yylloc, error);
}
return TokenType::CHINESE_LABEL;
return TokenType::LABEL;
}
. {
/**
Expand Down
17 changes: 15 additions & 2 deletions tests/tck/features/schema/Schema.feature
Original file line number Diff line number Diff line change
Expand Up @@ -772,9 +772,9 @@ Feature: Insert string vid of vertex and edge
# chinese tag without quote mark
When executing query:
"""
CREATE TAG 队伍( 名字 string);
CREATE TAG 队伍withoutQuote( 名字 string);
"""
Then a SyntaxError should be raised at runtime:
Then the execution should be successful
# chinese tag and chinese prop
When executing query:
"""
Expand Down Expand Up @@ -832,6 +832,19 @@ Feature: Insert string vid of vertex and edge
Then the result should be, in any order:
| Field | Type | Null | Default | Comment |
| "时间" | "timestamp" | "YES" | EMPTY | EMPTY |
When executing query:
"""
CREATE TAG `队伍 s2;`(`名s字ss1` string);
"""
Then the execution should be successful
# desc cn-en mixed tag
When executing query:
"""
DESCRIBE TAG `队伍 s2;`
"""
Then the result should be, in any order:
| Field | Type | Null | Default | Comment |
| "名s字ss1" | "string" | "YES" | EMPTY | EMPTY |
When executing query:
"""
DROP SPACE issue2009;
Expand Down