Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

implement JSON type in FE #3084

Merged
merged 11 commits into from
Jan 29, 2022
Merged

Conversation

murphyatwork
Copy link
Contributor

@murphyatwork murphyatwork commented Jan 25, 2022

What type of PR is this:

  • bug
  • feature: add json type in FE
  • enhancement
  • others

Modification

  • add keyword json for type
  • add json type in Type.java
  • add json type in thrift
  • support type casting for json
  • implement -> syntax for json_query function
  • JSON is not allowed used for partition by, distributed by, order by, group by ...

Not included

@murphyatwork murphyatwork mentioned this pull request Jan 26, 2022
4 tasks
@murphyatwork murphyatwork force-pushed the mofei_json_fe branch 2 times, most recently from fc57327 to 0a4fd6f Compare January 26, 2022 08:05
@murphyatwork murphyatwork changed the title implement JSON type in FE [WIP] implement JSON type in FE Jan 26, 2022
@murphyatwork murphyatwork changed the title [WIP] implement JSON type in FE implement JSON type in FE Jan 27, 2022
@@ -4288,6 +4293,11 @@ array_slice ::=
{: RESULT = new ArraySliceExpr(e, new IntLiteral(0), new IntLiteral(-1)); :}
;

arrow_expr ::=
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this arrow_expr neceesary for JOSN?
I'm a little worried if this definition will be conflict with lambda expression?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The arrow expression looks like '->', currently used a syntax sugar for JSON.

Current implementation of arrow expression is just translate into json_query($value, $path), which would conflict with some other usage like lambda expression.

However, I think it's quite easy to avoid conflicts, just look up the context of this expression, and translate it according to the context.

Anyway, some existing jsonpath sugar, like col->'field', col.k1.k2, col[k1][k2], are always has ambiguity with other SQL syntax, there's no simple approach to create an unique syntax.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I'm worried is that these two expr will cause shift/reduce conflict when parsing phase if we support lambda later.
One way is to create a super expr compatible with both ArorwExpr and Lambda.
Is it possible?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One possible way:

arrow_expr is defined as : left_expr '->' right_expr.

Resolve rule:

  1. If left_expr could be resolved a json type, including column_ref or function result, it's a json access expression. And the right_expr must be a literal string expression.
  2. Otherwise, left_expr could not be resolved, the whole expression is recognized as lambda expression

Which means, in parsing phase they could not be differentiated, only analyzer could tell the truth.

@kangkaisen kangkaisen merged commit 2b51852 into StarRocks:main Jan 29, 2022
jaogoy pushed a commit to jaogoy/starrocks that referenced this pull request Nov 15, 2023
* Update STREAM LOAD.md

* Update BROKER LOAD.md

* modified file_path descriptions in BROKERLOAD.md
@murphyatwork murphyatwork deleted the mofei_json_fe branch July 23, 2024 08:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants