[minor] Update docs for functions.scala to make it clear not all the built-in functions are defined there #21318
Conversation
|
Test build #90567 has finished for PR 21318 at commit
|
|
Hm the failure doesn't look like it's caused by this PR. Do you guys know what's going on? |
| * a little bit more compile-time safety to make sure the function exists. | ||
| * | ||
| * Spark also includes more built-in functions that are less common and are not defined here. | ||
| * You can still access them (and all the functions defined here) using the [[functions.expr()]] API |
There was a problem hiding this comment.
[error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/core/target/java/org/apache/spark/sql/functions.java:7: error: unexpected text
[error] * You can still access them (and all the functions defined here) using the {@link functions.expr()} API
[error] ^
[error] /home/jenkins/workspace/SparkPullRequestBuilder/sql/core/target/java/org/apache/spark/sql/functions.java:9: error: unexpected text
[error] * the latest version of Spark at {@link https://spark.apache.org/docs/latest/api/sql/index.html}.
[error] ^
Seems both links are the problem in Javadoc. Shall we just use `functions.expr() ` and leave the https://spark.apache.org/docs/latest/api/sql/index.html like without [[...]]?
| * Spark also includes more built-in functions that are less common and are not defined here. | ||
| * You can still access them (and all the functions defined here) using the [[functions.expr()]] API | ||
| * and calling them through a SQL expression string. You can find the entire list of functions for | ||
| * the latest version of Spark at [[https://spark.apache.org/docs/latest/api/sql/index.html]]. |
There was a problem hiding this comment.
@rxin, it's rather a nit but shouldn't we always update the link for each release since it always points the latest? Probably, just prose is fine.
There was a problem hiding this comment.
it's just a lot of work and i'm sure we will forget to update ... so i'm pointing to the latest.
|
|
||
| /** | ||
| * Functions available for DataFrame operations. | ||
| * Commonly used functions available for DataFrame operations. Using functions defined here provides |
There was a problem hiding this comment.
Maybe I am too much caring about this but I hope we don't have arguments too much about which function is common or not ...
| * the latest version of Spark at [[https://spark.apache.org/docs/latest/api/sql/index.html]]. | ||
| * | ||
| * As an example, `isnan` is a function that is defined here. You can use `isnan(col("myCol"))` | ||
| * to invoke the isnan function. This way the programming language's compiler ensures isnan exists |
|
@rxin, should we maybe we mention that SQL functions are usually added to match other DBMSs (or Hive) (unlike functions.scala)? |
| * As an example, `isnan` is a function that is defined here. You can use `isnan(col("myCol"))` | ||
| * to invoke the isnan function. This way the programming language's compiler ensures isnan exists | ||
| * and is of the proper form. You can also use `expr("isnan(myCol)")` function to invoke the same | ||
| * function. In this case, Spark itself will ensure isnan exists when it analyzes the query. |
|
retest this please |
|
It's still going to fail because I haven't updated it yet. Will do tomorrow. |
|
Test build #90625 has finished for PR 21318 at commit
|
|
Just browsing old PRs .. want to finish this one up @rxin? looks simple and useful. |
|
Yup will do.
|
|
Test build #93695 has finished for PR 21318 at commit
|
|
Thanks! Merged to master. |
|
@rxin re: #21318 (comment) I meant to say for instance, like "Please refer the SQL function documentation in the corresponding version". We don't have to bother update and also it makes sense .. |
|
LGTM.
…On Fri, Jul 27, 2018 at 10:58 PM Hyukjin Kwon ***@***.***> wrote:
@rxin <https://github.com/rxin> re: #21318 (comment)
<#21318 (comment)> I
meant to say for instance, like "Please refer the SQL function
documentation in the corresponding version". We don't have to bother update
and also it makes sense ..
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#21318 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AATvPO0G9cnPbJEyopz69kHMROglqL_Iks5uK_2TgaJpZM4T9LE->
.
|
6 participants
The title summarizes the change.