Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion sql/core/src/main/scala/org/apache/spark/sql/functions.scala
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,21 @@ import org.apache.spark.util.Utils


/**
* Functions available for DataFrame operations.
* Commonly used functions available for DataFrame operations. Using functions defined here provides
Copy link
Member

@HyukjinKwon HyukjinKwon May 14, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I am too much caring about this but I hope we don't have arguments too much about which function is common or not ...

* a little bit more compile-time safety to make sure the function exists.
*
* Spark also includes more built-in functions that are less common and are not defined here.
* You can still access them (and all the functions defined here) using the `functions.expr()` API
* and calling them through a SQL expression string. You can find the entire list of functions for
* the latest version of Spark at https://spark.apache.org/docs/latest/api/sql/index.html.
*
* As an example, `isnan` is a function that is defined here. You can use `isnan(col("myCol"))`
* to invoke the `isnan` function. This way the programming language's compiler ensures `isnan`
* exists and is of the proper form. You can also use `expr("isnan(myCol)")` function to invoke the
* same function. In this case, Spark itself will ensure `isnan` exists when it analyzes the query.
*
* `regr_count` is an example of a function that is built-in but not defined here, because it is
* less commonly used. To invoke it, use `expr("regr_count(yCol, xCol)")`.
*
* @groupname udf_funcs UDF functions
* @groupname agg_funcs Aggregate functions
Expand Down