Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

community: scaling xskillscore #404

Closed
raybellwaves opened this issue Apr 2, 2020 · 1 comment
Closed

community: scaling xskillscore #404

raybellwaves opened this issue Apr 2, 2020 · 1 comment

Comments

@raybellwaves
Copy link
Member

Hi dask-jobqueue,

I previously worked on #78

I recently showed how a package I developed works with dask (https://github.com/raybellwaves/xskillscore-tutorial/blob/master/03_Big_Data.ipynb).

Unfortunately, I don't have access to a HPC anymore and can't demonstrate how it scales beyond my laptop.

If you are interested I setup a friendly competition in the issue tracker to see how large people can scale the dataset https://github.com/raybellwaves/xskillscore-tutorial/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc

If you would like to partake you are welcome to run the example on an Cluster and you can even have a competition among the different schedulers :)

I'll close this in a week.

@lesteve
Copy link
Member

lesteve commented Apr 7, 2020

Thanks for letting us know and thanks a lot for the LSFCluster implementation! I met some people that were using LSFCluster at the Dask workshop in Washington a few months ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants