[SPARK-14088][SQL] Some Dataset API touch-up

[rxin]

What changes were proposed in this pull request?

1. Deprecated unionAll. It is pretty confusing to have both "union" and "unionAll" when the two do the same thing in Spark but are different in SQL.
2. Rename reduce in KeyValueGroupedDataset to reduceGroups so it is more consistent with rest of the functions in KeyValueGroupedDataset. Also makes it more obvious what "reduce" and "reduceGroups" mean. Previously it was confusing because it could be reducing a Dataset, or just reducing groups.
3. Added a "name" function, which is more natural to name columns than "as" for non-SQL users.
4. Remove "subtract" function since it is just an alias for "except".

How was this patch tested?

All changes should be covered by existing tests. Also added couple test cases to cover "name".

[rxin]

cc @davies for Python, and @cloud-fan / @liancheng for the rest.

[cloud-fan]

Is there any way to avoid duplicating documents at scala side?

[rxin]

no unfortunately...

[liancheng]

Well, actually there is. You can use @define <name> <definition> and then reference it using $name later. See here. Scala standard library uses this trick extensively.

[rxin]

does it work with the unidoc for java?

[liancheng]

Just checked, unfortunately it doesn't work for Java :(

[cloud-fan]

LGTM

[SparkQA]

Test build #53866 has finished for PR 11908 at commit 87c4641.

• This patch fails MiMa tests.
• This patch merges cleanly.
• This patch adds no public classes.

[davies]

LGTM for python part

[SparkQA]

Test build #53886 has finished for PR 11908 at commit a7e98b8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[rxin]

Thanks - merging in master.