[SPARK-4459] Change groupBy type parameter from K to U

[alokito]

Please see https://issues.apache.org/jira/browse/SPARK-4459

[AmplabJenkins]

Can one of the admins verify this patch?

[srowen]

LGTM. The question is whether this changes the binary or source compatibility. I don't think it changes source compatibility since it merely loosens a restriction. I don't think it would change the binary signature either? but MIMA will help know for sure.

Also, I think keyBy has the same issue?

[srowen]

(Minor: indentation)

[sryza]

partitionBy as well?

[alokito]

I may have time to add some tests and patch keyBy and partitionBy tomorrow.

[alokito]

I addressed keyBy, but I think partitionBy does not actually have this issue, as the Partitioner abstract class does not take a type parameter - implementations such as RangePartitioner use asInstanceOf in getPartition.

[JoshRosen]

It looks like you might have run into a Scala compiler bug, since in principle it shouldn't matter what we name that type parameter, since it's in a different scope than other type parameters. This looks a lot like https://issues.scala-lang.org/browse/SI-6057, which I hit while implementing the Java API and @tdas ran into while working on the Java API for Spark Streaming.

[JoshRosen]

@alokito,

If you've got time, do you mind fixing up the indentation / formatting issues in the test suite so that I can merge this? If not, just let me know and I'll do it myself.

[JoshRosen]

The spacing here is messy. Also, "" + x is messy; just do x.toString() instead if you want to convert an object in a string.

[JoshRosen]

I downloaded this and tested it out; everything looks fine, modulo these formatting issues. I've fixed the style issues myself and am going to merge this into master, branch-1.2, and branch-1.1. Thanks for fixing this!

(I ran the MiMa tests locally and this passes)

[alokito]

Thanks for fixing the style issues, I meant to but it's been a tough week.

On Dec 4, 2014, at 17:57, Josh Rosen notifications@github.com wrote:

I downloaded this and tested it out; everything looks fine, modulo these
formatting issues. I've fixed the style issues myself and am going to merge
this into master, branch-1.2, and branch-1.1. Thanks for fixing this!

(I ran the MiMa tests locally and this passes)

—
Reply to this email directly or view it on GitHub
#3327 (comment).