Update Dask backend

[scharlottej13]

Opening this PR to update the documentation for using the Dask backend, since it seems the last changes were in #613.

Noting this is also referenced in the scikit-learn docs. Here's a screenshot of the changed from building the sklearn docs locally:

cc @mrocklin @ncclementi @jrbourbeau

[codecov]

Codecov Report

Patch coverage has no change and project coverage change: +0.02 🎉

Comparison is base (5543c71) 94.80% compared to head (86df3af) 94.83%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1411      +/-   ##
==========================================
+ Coverage   94.80%   94.83%   +0.02%     
==========================================
  Files          45       45              
  Lines        6889     6889              
==========================================
+ Hits         6531     6533       +2     
+ Misses        358      356       -2     

Impacted Files	Coverage Δ
joblib/parallel.py	96.02% <ø> (ø)

... and 3 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

[ncclementi]

This looks good to me, I wonder who we should ping for a review.
It seems like the ci failures are unrelated to this PR.

Maybe a gentle ping to @GaelVaroquaux or @tomMoral

[jrbourbeau]

Thanks @scharlottej13 @ncclementi. I agree the failures in CI appear to be unrelated to the changes here. I'm checking in #1412 whether or not those failures are already present on the main branch

Also cc @ogrisel @lesteve in case either of you have bandwidth for a review

EDIT: The linting failure does look related

[jrbourbeau]

Okay, so the linting error is (clearly) related to the changes here. The linux_pypy3 build isn't failing in #1412, but also seems unrelated (at least to me naively). Maybe it's a transient error? @scharlottej13 would you mind merging the main branch and resolving that linting issue? That will trigger another CI build and we can see if the linux_pypy3 failure persists

[scharlottej13]

Thanks @jrbourbeau and @ncclementi!

Okay, so the linting error is (clearly) related to the changes here.

Yes, it seems to be happening here due to a long link (line 10). Unfortunately I'm not quite sure what the best fix is. I tried separating the link from the target, but the link didn't render correctly in the RTD build. I also tried ignoring the line with # noqua: E501, however, then that comment shows up in the documentation too.

The linux_pypy3 build isn't failing in #1412, but also seems unrelated (at least to me naively). Maybe it's a transient error? @scharlottej13 would you mind merging the main branch and resolving that linting issue? That will trigger another CI build and we can see if the linux_pypy3 failure persists

This test seems to be passing now! My branch (and fork) were both already up to date with main.

[lesteve]

This looks good.

Just curious, were they something in particular that prompted your PR, e.g. something that you thought was not up-to-date or missing?

Also, it would be nice to try to make the linter happy, I'll try to take a look (probably next week).

[scharlottej13]

Just curious, were they something in particular that prompted your PR, e.g. something that you thought was not up-to-date or missing?

Thanks for having a look @lesteve! Yes, the Deploy Dask Clusters was recently updated in the Dask docs (see dask/dask#9912, should have mentioned that in my initial comment).

Also, it would be nice to try to make the linter happy, I'll try to take a look (probably next week).

Appreciate your help with this.

[tomMoral]

Thx a lot for the PR.
a couple of comment but otherwise LGTM.

[scharlottej13]

Thanks @lesteve and @tomMoral for the review! I believe I've responded to your comments. Feel free to let me know if there's anything else.

[tomMoral]

LGTM, just did a small reorder of the backend to put dark first as it is the only one with an example.
Let's change the docstring example in a follow up PR.

[tomMoral]

thanks @scharlottej13 for the PR :)

[scharlottej13]

Of course, thanks @tomMoral!