[SPARK-11605] [MLlib] ML 1.6 QA: API: Java compatibility, docs

[hhbyyh]

jira: https://issues.apache.org/jira/browse/SPARK-11605
Check Java compatibility for MLlib for this release.

fix:

1. StreamingTest.registerStream needs java friendly interface.
2. GradientBoostedTreesModel.computeInitialPredictionAndError and GradientBoostedTreesModel.updatePredictionError has java compatibility issue. Mark them as developerAPI.

TBD:
[updated] no fix for now per discussion.
org.apache.spark.mllib.classification.LogisticRegressionModel
public scala.Option<java.lang.Object> getThreshold(); has wrong return type for Java invocation.
SVMModel has the similar issue.

Yet adding a scala.Option<java.util.Double> getThreshold() would result in an overloading error due to the same function signature. And adding a new function with different name seems to be not necessary.

cc @jkbradley @feynmanliang

[SparkQA]

Test build #47060 has finished for PR 10102 at commit 5550a97.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[hhbyyh]

This can be Int to support multiClass sample data, right?

[jkbradley]

This was not part of the design, but I agree it would be nice someday. I'll ping @mengxr since he reviewed the original PRs, but I think we'll keep it as is for now.

[jkbradley]

I'll take a look now

[jkbradley]

I wouldn't worry about the getThreshold issue. I haven't seen users complain about it, and they will hopefully switch over to the spark.ml API anyways.

Can you please modify the old registerStream taking tuples to use BinarySample as well? We should be consistent across Scala and Java.

[jkbradley]

organize imports

[SparkQA]

Test build #47316 has finished for PR 10102 at commit 5077aa7.

• This patch fails Java style tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * public class JavaQuantileDiscretizerExample\n

[hhbyyh]

hudson.plugins.git.GitException: Failed to fetch from https://github.com/apache/spark.git
at hudson.plugins.git.GitSCM.fetchFrom(GitSCM.java:763)
at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1012)

need a retest

[hhbyyh]

@jkbradley Thanks for the review. Updated.

[jkbradley]

Thank you for updating! LGTM pending tests.

[SparkQA]

Test build #2185 has finished for PR 10102 at commit 5ef6ff8.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[jkbradley]

Merging with master and branch-1.6