Skip to content
This repository has been archived by the owner on Feb 10, 2021. It is now read-only.

Subclass distributed.Adaptive #35

Merged
merged 2 commits into from
Aug 29, 2017

Conversation

TomAugspurger
Copy link
Member

This is still a work in progress. I still have a few pieces that were previously all in ._adapt that need to be added back in. This depends on dask/distributed#1311

  • jobs with queued_active (I think move to should_scale_up)
  • len(s.workers) < len(self.cluster.workers)
  • Starting up busy number of clusters, instead of 2 * the current (from distributed.Adaptive.get_scale_up_kwargs

@TomAugspurger
Copy link
Member Author

@mrocklin I'm not sure what kind of backwards compatibility guarantees we make here, but distributed.Adaptive takes (scheduler, cluster), while dask_drmaa.Adaptive takes (cluster, scheduler). I think we should align on distributed's signature.

5fd79bc warns if the arguments are in the incorrect order.

@TomAugspurger TomAugspurger changed the title WIP: subclass distributed.Adaptive Subclass distributed.Adaptive Aug 28, 2017
@mrocklin
Copy link
Member

Adaptive isn't used much. I encourage a move-fast-and-break-things approach.

@TomAugspurger TomAugspurger merged commit 195acbb into dask:master Aug 29, 2017
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants