Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specification of overhead per worker and memory allocation for dask app causes error in low resource machines #119

Open
jackransomlovell opened this issue Aug 14, 2022 · 0 comments

Comments

@jackransomlovell
Copy link

When using the train-pca CLI for the out-of-the-box small dataset example I ran into the following error:

$ moseq2-pca train-pca -i aggregate_results/ -o _pca_tst/ --config-file config_tst.yaml
...

File "/Users/jacklovell/opt/anaconda3/envs/moseq2-app/lib/python3.7/site-packages/moseq2_pca/util.py", line 548, in initialize_dask
    mem_limit = max(1, max_mem / nworkers)
ZeroDivisionError: float division by zero

After investigating, this is caused by allocating 0.4 of the max memory available (line 537), making the max number of workers 0 (which is set to nworkers). The error thrown was also a bit informative.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant