Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] document valid config options for ECS / Fargate #156

Merged
merged 3 commits into from
Nov 9, 2020
Merged

[docs] document valid config options for ECS / Fargate #156

merged 3 commits into from
Nov 9, 2020

Conversation

jameslamb
Copy link
Member

I've been using dask-cloudprovider a lot recently to test things and come up with reproducible examples.

Through this experience, I learned that ECS / Fargate only allow task definitions to choose from a finite list of CPU and memory settings. This is an AWS limitation and not a dask-cloudprovider one, but it took me a bit of investigation to figure that out.

botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: No Fargate configuration exists for given values.

This PR proposes adding a link to the valid config values for ECS in the docs, so that when people search the dask-cloudprovider docs or Google for this error, they quickly find the answer and the set of valid values.

Thanks for your time and consideration.

Copy link
Member

@jacobtomlinson jacobtomlinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great thanks @jameslamb.

Should we also mention this (or link to the troubleshooting guide) from the docstrings that describe these kwargs.

scheduler_cpu: int (optional)
The amount of CPU to request for the scheduler in milli-cpu (1/1024).
Defaults to ``1024`` (one vCPU).
scheduler_mem: int (optional)
The amount of memory to request for the scheduler in MB.
Defaults to ``4096`` (4GB).

worker_cpu: int (optional)
The amount of CPU to request for worker tasks in milli-cpu (1/1024).
Defaults to ``4096`` (four vCPUs).
worker_mem: int (optional)
The amount of memory to request for worker tasks in MB.
Defaults to ``16384`` (16GB).

@jameslamb
Copy link
Member Author

oh sure, good idea! I'll link to the troubleshooting guide from there, to avoid duplicating that AWS link into multiple places.

@jameslamb
Copy link
Member Author

Alright I just pushed an update, let me know what you think! I regenerated the docs like this:

pip install .[digitalocean]
cd doc
make html

Locally, after that change the link looks like this:

image

And I clicked through and confirmed that the relevant reference like /troubleshooting.html#invalid-cpu-or-memory works as expected.

@jacobtomlinson
Copy link
Member

Perfect thanks @jameslamb

@jacobtomlinson jacobtomlinson merged commit fad780e into dask:master Nov 9, 2020
@jacobtomlinson jacobtomlinson added the documentation Improvements or additions to documentation label Nov 9, 2020
@jameslamb jameslamb deleted the docs/ecs-config branch November 9, 2020 16:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants