Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move dask-gateway workers to their own nodegroups on the openscapes cluster #4485

Closed
6 tasks done
Tracked by #4453
yuvipanda opened this issue Jul 24, 2024 · 0 comments · Fixed by #4519
Closed
6 tasks done
Tracked by #4453

Move dask-gateway workers to their own nodegroups on the openscapes cluster #4485

yuvipanda opened this issue Jul 24, 2024 · 0 comments · Fixed by #4519
Assignees

Comments

@yuvipanda
Copy link
Member

yuvipanda commented Jul 24, 2024

Similar to #4482 but for dask-gateway.

  • Create the same set of dask worker nodes for staging, prod and workshop
  • Configure each of the hubs so their dask workers only spawn on the nodepools designated for them
  • Tag each of the nodepool with 2i2c:hub-name to match the name of the hub
  • Tag each of the dask nodepools with 2i2c:node-purpose set to worker

Definition of done

  • Users across these hubs will spawn dask workers on to their own nodepools, and not share them with users on other hubs.
  • There is a tag 2i2c:hub-name, 2i2c:node-purpose on nodes spawned by dask-gateway. You can verify this by looking at EC2 instances on the AWS console.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants