SPARK-4040. Update documentation to exemplify use of local (n) value, fo...

[jayunit100]

This is a minor docs update which helps to clarify the way local[n] is used for streaming apps.

[AmplabJenkins]

Can one of the admins verify this patch?

[jayunit100]

(bump) - any thoughts on this ? I'd also like role some more improvements into it in a follow up......

[srowen]

I actually though the default behavior of local was 2 threads, but in the code I don't think that's true. I thought @mateiz mentioned one time that it's better to run with minimal parallelism by default to expose issues that might only appear when there are multiple executors.

In any event, given that, and the thrust of this doc change, is it good to encourage people to use 1 worker? how about explicitly 2? Making it explicit is a small good thing anyway.

[jayunit100]

Hi sean . I like that idea of running with 2 threads , and making it explicit : Thats the main purpose of the PR.... i'll update that (and the // stuff) , and then rebase this PR

[jayunit100]

okay ! updated . After this i think we can look into some deeper updates into the streaming docs as well. (fyi @srowen ) looking good now?

[mateiz]

One note on this, the threads shouldn't be called "workers", since that means something else in our distributed cluster mode. It's better to call them threads here.

Also, on this like, capitalize Spark Streaming.

[mateiz]

Thanks for adding these clarifications, it's a good idea.

[jayunit100]

@mateiz @srowen okay, updated w/ the threads vs workers disambiguation... thanks for the feedback, just ping if any other updates are necessary.!

[jayunit100]

(bump) all set on this guy ? or shall we wait till after the upcoming spark release?

[mateiz]

This looks fine to merge into 1.2; will do so. Thanks!