feat(function): Support Semi-structured function GET/GET_IGNORE_CASE/GET_PATH #4684

b41sh · 2022-04-04T02:53:53Z

I hereby agree to the terms of the CLA available at: https://databend.rs/dev/policies/cla/

Summary

Support GET function
Support GET_IGNORE_CASE function
Support GET_PATH function

Changelog

New Feature
Build/Testing/CI
Documentation

Related Issues

Fixes #4665

Test Plan

Unit Tests

Stateless Tests

vercel · 2022-04-04T02:53:58Z

This pull request is being automatically deployed with Vercel (learn more).
To see the status of your deployment, click below or on the icon next to each commit.

🔍 Inspect: https://vercel.com/databend/databend/6L2CxR4bqkHs7FoJ2MciC3NWeUyb
✅ Preview: https://databend-git-fork-b41sh-func-get-databend.vercel.app

mergify · 2022-04-04T02:54:25Z

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

BohuTANG · 2022-04-06T01:05:34Z

@mergify update

mergify · 2022-04-06T01:05:48Z

update

✅ Branch has been successfully updated

Hey, I reacted but my real name is @Mergifyio

common/functions/src/scalars/semi_structureds/get.rs

BohuTANG · 2022-04-06T05:29:06Z



SyntaxError: /vercel/path0/docs/doc/30-reference/20-functions/110-semi-structured-functions/get_path.md: Expected corresponding JSX closing tag for <index>. (10:409)
--
10:23:22.794 | 8 \| <p>{`Extracts value from an `}<inlineCode parentName="p">{`ARRAY`}</inlineCode>{`, an `}<inlineCode parentName="p">{`OBJECT`}</inlineCode>{`, or a `}<inlineCode parentName="p">{`VARIANT`}</inlineCode>{` by `}<inlineCode parentName="p">{`path_name`}</inlineCode>{`.
10:23:22.794 | 9 \| The value is returned as a `}<inlineCode parentName="p">{`Variant`}</inlineCode>{` or `}<inlineCode parentName="p">{`NULL`}</inlineCode>{` if either of the arguments is `}<inlineCode parentName="p">{`NULL`}</inlineCode>{`.`}</p>
10:23:22.794 | > 10 \| <p><inlineCode parentName="p">{`GET_PATH`}</inlineCode>{` is equivalent to a chain of `}<inlineCode parentName="p">{`GET`}</inlineCode>{` functions, `}<inlineCode parentName="p">{`path_name`}</inlineCode>{` consists of a concatenation of field names preceded by periods (.), colons (:) or index operators (`}{`[`}<index>{`]`}{`). The first field name does not require the leading identifier to be specified.`}</p>
10:23:22.794 | \|                                                                                                                                                                                                                                                                                                                                                                                                                          ^
10:23:22.794 | 11 \| <h2 {...{"id":"syntax"}}>{`Syntax`}</h2>
10:23:22.794 | 12 \| <pre><code parentName="pre" {...{"className":"language-sql"}}>{`get_path(array, path_name)
10:23:22.794 | 13 \| get_path(object, path_name)
10:23:22.795 | [ERROR] Client bundle compiled with errors therefore further build is impossible.
10:23:22.843 | error Command failed with exit code 1.
10:23:22.844 | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
10:23:22.852 | Error: Command "yarn run build" exited with 1

b41sh · 2022-04-06T05:52:19Z



SyntaxError: /vercel/path0/docs/doc/30-reference/20-functions/110-semi-structured-functions/get_path.md: Expected corresponding JSX closing tag for <index>. (10:409)
--
10:23:22.794 | 8 \| <p>{`Extracts value from an `}<inlineCode parentName="p">{`ARRAY`}</inlineCode>{`, an `}<inlineCode parentName="p">{`OBJECT`}</inlineCode>{`, or a `}<inlineCode parentName="p">{`VARIANT`}</inlineCode>{` by `}<inlineCode parentName="p">{`path_name`}</inlineCode>{`.
10:23:22.794 | 9 \| The value is returned as a `}<inlineCode parentName="p">{`Variant`}</inlineCode>{` or `}<inlineCode parentName="p">{`NULL`}</inlineCode>{` if either of the arguments is `}<inlineCode parentName="p">{`NULL`}</inlineCode>{`.`}</p>
10:23:22.794 | > 10 \| <p><inlineCode parentName="p">{`GET_PATH`}</inlineCode>{` is equivalent to a chain of `}<inlineCode parentName="p">{`GET`}</inlineCode>{` functions, `}<inlineCode parentName="p">{`path_name`}</inlineCode>{` consists of a concatenation of field names preceded by periods (.), colons (:) or index operators (`}{`[`}<index>{`]`}{`). The first field name does not require the leading identifier to be specified.`}</p>
10:23:22.794 | \|                                                                                                                                                                                                                                                                                                                                                                                                                          ^
10:23:22.794 | 11 \| <h2 {...{"id":"syntax"}}>{`Syntax`}</h2>
10:23:22.794 | 12 \| <pre><code parentName="pre" {...{"className":"language-sql"}}>{`get_path(array, path_name)
10:23:22.794 | 13 \| get_path(object, path_name)
10:23:22.795 | [ERROR] Client bundle compiled with errors therefore further build is impossible.
10:23:22.843 | error Command failed with exit code 1.
10:23:22.844 | info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
10:23:22.852 | Error: Command "yarn run build" exited with 1

done.

mergify · 2022-04-07T02:05:59Z

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

mergify · 2022-04-07T02:08:06Z

Thanks for the contribution!
I have applied any labels matching special text in your PR Changelog.

Please review the labels and make any necessary changes.

common/functions/src/scalars/semi_structureds/get.rs

sundy-li · 2022-04-07T04:13:59Z

common/functions/src/scalars/semi_structureds/get.rs

+            continue;
+        }
+        for v in column.iter() {
+            let mut found_value = true;


Will found_value defaults to false make this more simple?

The default value of found_value is not simpler to set as false, because the lookup is recursive. If we can find a value we need to set it to true, otherwise we set it to false. If the default value is false, the following set found_value = false can't be omitted, but need to add some code to set found_value = true.

BohuTANG · 2022-04-07T05:05:06Z

+++ /workspace/tests/suites/0_stateless/02_function/02_0051_function_semi_structureds_get.stdout	2022-04-07 05:02:22.259013252 +0000
@@ -1,3 +1,18 @@
+ERROR 2013 (HY000) at line 33: Lost connection to MySQL server during query
+ERROR 2006 (HY000) at line 34: MySQL server has gone away
+ERROR 2006 (HY000) at line 35: MySQL server has gone away
+ERROR 2006 (HY000) at line 36: MySQL server has gone away
+ERROR 2006 (HY000) at line 38: MySQL server has gone away
+ERROR 2006 (HY000) at line 39: MySQL server has gone away
+ERROR 2006 (HY000) at line 40: MySQL server has gone away
+ERROR 2006 (HY000) at line 42: MySQL server has gone away
+ERROR 2006 (HY000) at line 43: MySQL server has gone away
+ERROR 2006 (HY000) at line 44: MySQL server has gone away
+ERROR 2006 (HY000) at line 45: MySQL server has gone away
+ERROR 2006 (HY000) at line 46: MySQL server has gone away
+ERROR 2006 (HY000) at line 47: MySQL server has gone away
+ERROR 2006 (HY000) at line 48: MySQL server has gone away
+ERROR 2006 (HY000) at line 50: MySQL server has gone away
 ==get==
 2.71
 NULL
@@ -13,18 +28,4 @@
 "databend"
 "ext"
 NULL
-+==get from table==
-1
-NULL
-NULL
-1
-==get_ignore_case from table==
-1
-1
-==get_path from table==
-1
-"a"
-1
-1
-2

b41sh · 2022-04-07T06:04:49Z

+++ /workspace/tests/suites/0_stateless/02_function/02_0051_function_semi_structureds_get.stdout	2022-04-07 05:02:22.259013252 +0000
@@ -1,3 +1,18 @@
+ERROR 2013 (HY000) at line 33: Lost connection to MySQL server during query
+ERROR 2006 (HY000) at line 34: MySQL server has gone away
+ERROR 2006 (HY000) at line 35: MySQL server has gone away
+ERROR 2006 (HY000) at line 36: MySQL server has gone away
+ERROR 2006 (HY000) at line 38: MySQL server has gone away
+ERROR 2006 (HY000) at line 39: MySQL server has gone away
+ERROR 2006 (HY000) at line 40: MySQL server has gone away
+ERROR 2006 (HY000) at line 42: MySQL server has gone away
+ERROR 2006 (HY000) at line 43: MySQL server has gone away
+ERROR 2006 (HY000) at line 44: MySQL server has gone away
+ERROR 2006 (HY000) at line 45: MySQL server has gone away
+ERROR 2006 (HY000) at line 46: MySQL server has gone away
+ERROR 2006 (HY000) at line 47: MySQL server has gone away
+ERROR 2006 (HY000) at line 48: MySQL server has gone away
+ERROR 2006 (HY000) at line 50: MySQL server has gone away
 ==get==
 2.71
 NULL
@@ -13,18 +28,4 @@
 "databend"
 "ext"
 NULL
-+==get from table==
-1
-NULL
-NULL
-1
-==get_ignore_case from table==
-1
-1
-==get_path from table==
-1
-"a"
-1
-1
-2

this is caused by another bug, I will fix it.

BohuTANG · 2022-04-08T02:59:12Z

Conflicts that must be resolved :D

…, get_path

b41sh requested a review from BohuTANG as a code owner April 4, 2022 02:53

databend-bot added the need-review label Apr 4, 2022

vercel bot had a problem deploying to Preview April 4, 2022 02:53 Failure

mergify bot added pr-feature this PR introduces a new feature to the codebase pr-build this PR changes build/testing/ci steps pr-doc-fix labels Apr 4, 2022

b41sh requested a review from sundy-li April 4, 2022 02:54

vercel bot had a problem deploying to Preview April 6, 2022 01:05 Failure

b41sh force-pushed the func-get branch from 52f7105 to a1e44af Compare April 6, 2022 01:58

vercel bot had a problem deploying to Preview April 6, 2022 01:59 Failure

vercel bot had a problem deploying to Preview April 6, 2022 02:13 Failure

b41sh force-pushed the func-get branch from fb539b1 to 2a6ba87 Compare April 6, 2022 02:15

vercel bot had a problem deploying to Preview April 6, 2022 02:16 Failure

b41sh force-pushed the func-get branch from 2a6ba87 to e822aa3 Compare April 6, 2022 02:21

vercel bot had a problem deploying to Preview April 6, 2022 02:21 Failure

sundy-li reviewed Apr 6, 2022

View reviewed changes

common/functions/src/scalars/semi_structureds/get.rs Outdated Show resolved Hide resolved

b41sh force-pushed the func-get branch from e822aa3 to fa7d1ea Compare April 6, 2022 05:51

vercel bot deployed to Preview April 6, 2022 05:51 View deployment

b41sh marked this pull request as draft April 6, 2022 17:30

b41sh force-pushed the func-get branch from fa7d1ea to 96c027a Compare April 7, 2022 01:44

vercel bot deployed to Preview April 7, 2022 01:44 View deployment

b41sh force-pushed the func-get branch from 96c027a to 5594b89 Compare April 7, 2022 02:04

vercel bot deployed to Preview April 7, 2022 02:04 View deployment

b41sh marked this pull request as ready for review April 7, 2022 02:07

sundy-li reviewed Apr 7, 2022

View reviewed changes

common/functions/src/scalars/semi_structureds/get.rs Outdated Show resolved Hide resolved

sundy-li reviewed Apr 7, 2022

View reviewed changes

common/functions/src/scalars/semi_structureds/get.rs Outdated Show resolved Hide resolved

b41sh force-pushed the func-get branch from 5594b89 to 6ea9be4 Compare April 7, 2022 03:37

vercel bot deployed to Preview April 7, 2022 03:37 View deployment

b41sh force-pushed the func-get branch from 6ea9be4 to b582a73 Compare April 7, 2022 04:04

vercel bot deployed to Preview April 7, 2022 04:04 View deployment

sundy-li reviewed Apr 7, 2022

View reviewed changes

sundy-li approved these changes Apr 7, 2022

View reviewed changes

b41sh mentioned this pull request Apr 7, 2022

fix(function): Const(Nullable(Object)) column downcast to Nullable failed #4733

Merged

feat(function): Support Semi-structured function get, get_ignore_case…

502975e

…, get_path

b41sh force-pushed the func-get branch from b582a73 to 502975e Compare April 8, 2022 04:52

vercel bot deployed to Preview April 8, 2022 04:52 View deployment

BohuTANG merged commit 9a9589b into databendlabs:main Apr 8, 2022

BohuTANG mentioned this pull request May 12, 2022

Release proposal: Nightly v0.8 #4591

Closed

55 tasks

Xuanwo added this to the v0.8 milestone May 20, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(function): Support Semi-structured function GET/GET_IGNORE_CASE/GET_PATH #4684

feat(function): Support Semi-structured function GET/GET_IGNORE_CASE/GET_PATH #4684

b41sh commented Apr 4, 2022 •

edited

Loading

vercel bot commented Apr 4, 2022 •

edited

Loading

mergify bot commented Apr 4, 2022

BohuTANG commented Apr 6, 2022

mergify bot commented Apr 6, 2022

BohuTANG commented Apr 6, 2022

b41sh commented Apr 6, 2022

mergify bot commented Apr 7, 2022

mergify bot commented Apr 7, 2022

sundy-li Apr 7, 2022 •

edited

Loading

b41sh Apr 7, 2022 •

edited

Loading

BohuTANG commented Apr 7, 2022

b41sh commented Apr 7, 2022

BohuTANG commented Apr 8, 2022

feat(function): Support Semi-structured function GET/GET_IGNORE_CASE/GET_PATH #4684

feat(function): Support Semi-structured function GET/GET_IGNORE_CASE/GET_PATH #4684

Conversation

b41sh commented Apr 4, 2022 • edited Loading

Summary

Changelog

Related Issues

Test Plan

vercel bot commented Apr 4, 2022 • edited Loading

mergify bot commented Apr 4, 2022

BohuTANG commented Apr 6, 2022

mergify bot commented Apr 6, 2022

✅ Branch has been successfully updated

BohuTANG commented Apr 6, 2022

b41sh commented Apr 6, 2022

mergify bot commented Apr 7, 2022

mergify bot commented Apr 7, 2022

sundy-li Apr 7, 2022 • edited Loading

Choose a reason for hiding this comment

b41sh Apr 7, 2022 • edited Loading

Choose a reason for hiding this comment

BohuTANG commented Apr 7, 2022

b41sh commented Apr 7, 2022

BohuTANG commented Apr 8, 2022

b41sh commented Apr 4, 2022 •

edited

Loading

vercel bot commented Apr 4, 2022 •

edited

Loading

sundy-li Apr 7, 2022 •

edited

Loading

b41sh Apr 7, 2022 •

edited

Loading