[SPARK-20403][SQL]Modify the instructions of some functions #17698

10110346 · 2017-04-20T07:23:16Z

What changes were proposed in this pull request?

add instructions of 'cast' function When using 'show functions' and 'desc function cast'
command in spark-sql
Modify the instructions of functions，such as
boolean，tinyint，smallint，int，bigint，float，double，decimal，date，timestamp，binary，string

How was this patch tested?

Before modification：
spark-sql>desc function boolean;
Function: boolean
Class: org.apache.spark.sql.catalyst.expressions.Cast
Usage: boolean(expr AS type) - Casts the value expr to the target data type type.

After modification：
spark-sql> desc function boolean;
Function: boolean
Class: org.apache.spark.sql.catalyst.expressions.Cast
Usage: boolean(expr) - Casts the value expr to the target data type boolean.

spark-sql> desc function cast
Function: cast
Class: org.apache.spark.sql.catalyst.expressions.Cast
Usage: cast(expr AS type) - Casts the value expr to the target data type type.

rxin · 2017-04-20T07:27:49Z

sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala

will this work with our pattern matches in the query optimizer?

No,it will not

srowen · 2017-04-20T07:28:13Z

core/src/main/scala/org/apache/spark/ui/jobs/UIData.scala

Revert these unrelated changes

has reverted

srowen · 2017-04-20T07:28:17Z

examples/src/main/scala/org/apache/spark/examples/LocalKMeans.scala

Revert these unrelated changes

has reverted

10110346 · 2017-05-06T06:07:23Z

I have tested all of them(boolean，tinyint，smallint，int，bigint，float，double，decimal，date，timestamp，binary，string), they can work properly. @srowen @rxin @HyukjinKwon

HyukjinKwon

For me, I don't think this is a proper approach (adding classes to fix a documentation).

HyukjinKwon · 2017-05-10T14:48:30Z

Would there be other better ways, for example, fixing it in DescribeFunctionCommand or just hijacking the usage within castAlias?

10110346 · 2017-05-11T00:57:01Z

@HyukjinKwon I agree with you,i will try ,thanks.

10110346 · 2017-05-11T02:51:13Z

I have updated it , and test passed
please review it again,thanks @srowen @rxin @HyukjinKwon

gatorsmile · 2017-05-18T18:45:47Z

LGTM except one comment:

Please add the test cases to the end of file cast.sql

DESC FUNCTION boolean;
DESC FUNCTION EXTENDED boolean;

You can generate the result file by

SPARK_GENERATE_GOLDEN_FILES=1 build/sbt "sql/test-only *SQLQueryTestSuite"

10110346 · 2017-05-19T09:03:59Z

@gatorsmile I have added test cases to the file cast.sql ,
could you help trigger tests
thanks.

kiszk · 2017-05-19T12:47:50Z

Jenkins, test this please

10110346 · 2017-05-22T07:08:38Z

@ueshin @kiszk Test is not started, could you help trigger it again?

gatorsmile · 2017-05-22T20:46:40Z

ok to test

SparkQA · 2017-05-22T22:26:55Z

Test build #77201 has finished for PR 17698 at commit e09dc2a.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Signed-off-by: liuxian <liu.xian3@zte.com.cn>

command in spark-sql Signed-off-by: liuxian <liu.xian3@zte.com.cn>

SparkQA · 2017-05-23T02:27:06Z

Test build #77213 has finished for PR 17698 at commit 6ce4220.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-05-23T03:07:42Z

Test build #77214 has finished for PR 17698 at commit 7edfed5.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2017-05-23T04:45:05Z

Test build #77217 has finished for PR 17698 at commit 10be7eb.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

ueshin · 2017-05-23T05:25:14Z

@10110346 Hi, you can use the command @gatorsmile mentioned above to generate the result file.

boolean，tinyint，smallint，int，bigint，float，double，decimal，date，timestamp，binary，string Signed-off-by: liuxian <liu.xian3@zte.com.cn>

SparkQA · 2017-05-23T10:23:05Z

Test build #77241 has finished for PR 17698 at commit 51949d7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

10110346 · 2017-05-23T22:05:34Z

Test passed, thanks. @ueshin @gatorsmile

gatorsmile · 2017-05-25T00:31:41Z

LGTM

## What changes were proposed in this pull request? 1. add instructions of 'cast' function When using 'show functions' and 'desc function cast' command in spark-sql 2. Modify the instructions of functions，such as boolean，tinyint，smallint，int，bigint，float，double，decimal，date，timestamp，binary，string ## How was this patch tested? Before modification： spark-sql>desc function boolean; Function: boolean Class: org.apache.spark.sql.catalyst.expressions.Cast Usage: boolean(expr AS type) - Casts the value `expr` to the target data type `type`. After modification： spark-sql> desc function boolean; Function: boolean Class: org.apache.spark.sql.catalyst.expressions.Cast Usage: boolean(expr) - Casts the value `expr` to the target data type `boolean`. spark-sql> desc function cast Function: cast Class: org.apache.spark.sql.catalyst.expressions.Cast Usage: cast(expr AS type) - Casts the value `expr` to the target data type `type`. Author: liuxian <liu.xian3@zte.com.cn> Closes #17698 from 10110346/wip_lx_0418. (cherry picked from commit 197f901) Signed-off-by: Xiao Li <gatorsmile@gmail.com>

gatorsmile · 2017-05-25T00:32:44Z

Thanks! Merging to master/2.2

rxin reviewed Apr 20, 2017

View reviewed changes

srowen requested changes Apr 20, 2017

View reviewed changes

10110346 force-pushed the wip_lx_0418 branch 2 times, most recently from 3d66dcd to 240a989 Compare April 25, 2017 01:34

10110346 changed the title ~~[SPARK-20403][SQL][Documentation]Modify the instructions of some functions, and add instructions of 'cast' function~~ [SPARK-20403][SQL][Documentation]Modify the instructions of some functions May 6, 2017

10110346 force-pushed the wip_lx_0418 branch from 240a989 to a820fc3 Compare May 10, 2017 02:08

10110346 changed the title ~~[SPARK-20403][SQL][Documentation]Modify the instructions of some functions~~ [SPARK-20403][SQL]Modify the instructions of some functions May 10, 2017

HyukjinKwon reviewed May 10, 2017

View reviewed changes

10110346 force-pushed the wip_lx_0418 branch from a820fc3 to 8e4f7cb Compare May 11, 2017 02:36

10110346 force-pushed the wip_lx_0418 branch from 8e4f7cb to e09dc2a Compare May 19, 2017 05:53

10110346 added 2 commits May 23, 2017 08:58

modify annotation:HALF_UP -> HALF_EVEN

57f2450

Signed-off-by: liuxian <liu.xian3@zte.com.cn>

add 'cast' function When using 'show functions' and 'desc function cast'

343611f

command in spark-sql Signed-off-by: liuxian <liu.xian3@zte.com.cn>

10110346 force-pushed the wip_lx_0418 branch 2 times, most recently from 6ce4220 to 7edfed5 Compare May 23, 2017 01:41

10110346 force-pushed the wip_lx_0418 branch from 7edfed5 to 10be7eb Compare May 23, 2017 03:13

Fix:Modify the usage of functions，such as

51949d7

boolean，tinyint，smallint，int，bigint，float，double，decimal，date，timestamp，binary，string Signed-off-by: liuxian <liu.xian3@zte.com.cn>

10110346 force-pushed the wip_lx_0418 branch from 10be7eb to 51949d7 Compare May 23, 2017 08:03

asfgit closed this in 197f901 May 25, 2017

[SPARK-20403][SQL]Modify the instructions of some functions #17698

[SPARK-20403][SQL]Modify the instructions of some functions #17698

Uh oh!

Conversation

10110346 commented Apr 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

How was this patch tested?

Uh oh!

rxin Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

10110346 Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

srowen Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

10110346 Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

srowen Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

10110346 Apr 20, 2017

Choose a reason for hiding this comment

Uh oh!

10110346 commented May 6, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HyukjinKwon left a comment

Choose a reason for hiding this comment

Uh oh!

HyukjinKwon commented May 10, 2017

Uh oh!

10110346 commented May 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

10110346 commented May 11, 2017

Uh oh!

gatorsmile commented May 18, 2017

Uh oh!

10110346 commented May 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kiszk commented May 19, 2017

Uh oh!

10110346 commented May 22, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gatorsmile commented May 22, 2017

Uh oh!

SparkQA commented May 22, 2017

Uh oh!

SparkQA commented May 23, 2017

Uh oh!

SparkQA commented May 23, 2017

Uh oh!

SparkQA commented May 23, 2017

Uh oh!

ueshin commented May 23, 2017

Uh oh!

SparkQA commented May 23, 2017

Uh oh!

10110346 commented May 23, 2017

Uh oh!

gatorsmile commented May 25, 2017

Uh oh!

gatorsmile commented May 25, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

10110346 commented Apr 20, 2017 •

edited

Loading

10110346 commented May 6, 2017 •

edited

Loading

10110346 commented May 11, 2017 •

edited

Loading

10110346 commented May 19, 2017 •

edited

Loading

10110346 commented May 22, 2017 •

edited

Loading