[SPARK-3550][MLLIB] Disable automatic rdd caching for relevant learners.

[staple]

The NaiveBayes, ALS, and DecisionTree learners do not require external caching to prevent repeated RDD re-evaluation during learning iterations. NaiveBayes only evaluates its input RDD once, while ALS and DecisionTree internally persist transformations of their input RDDs.

[SparkQA]

Can one of the admins verify this patch?

[mengxr]

add to whitelist

[mengxr]

this is ok to test

[SparkQA]

QA tests have started for PR 2412 at commit c8ff120.

• This patch merges cleanly.

[SparkQA]

QA tests have finished for PR 2412 at commit c8ff120.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class NonASCIICharacterChecker extends ScalariformChecker

[davies]

@staple I also addressed this in #2378 , could you help to review this part?

[davies]

@staple could you rebase this PR?

[staple]

@davies It looks like in your #2378 you already disabled caching for NaiveBayes and DecisionTree. The only difference from this patch is that I disabled caching for ALS as well.

We discussed this a bit here: #2378 (comment). I filed SPARK-3550 as a follow up of the work on uncached input warnings (#2347). The warnings are only supposed to be printed if the input data is accessed repeatedly on many iterations during learning. That's not the case with ALS, so a warning shouldn't be printed there. But I can see there's a case for caching because the input data is accessed not once but twice when constructing an intermediate representation of the data. I don't have a strong preference on whether we should or should not cache in python for the ALS learner.

If you are fine with continuing to cache in python for ALS, then there's no more work to be done for this ticket, SPARK-3550.

[davies]

@staple thanks, I'd like to keep it as before for ALS, could you close this PR (maybe also the issue)?

[staple]

@davies, sure will do