Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelize functions in test modules #557

Merged
merged 6 commits into from
Apr 25, 2020
Merged

Conversation

jdebacker
Copy link
Member

This PR updates test modules with longer running tests to set up and use a dask client with multiple workers.

@codecov-io
Copy link

codecov-io commented Apr 23, 2020

Codecov Report

Merging #557 into master will decrease coverage by 0.23%.
The diff coverage is 64.83%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #557      +/-   ##
==========================================
- Coverage   80.48%   80.24%   -0.24%     
==========================================
  Files          47       47              
  Lines        6420     6500      +80     
==========================================
+ Hits         5167     5216      +49     
- Misses       1253     1284      +31     
Impacted Files Coverage Δ
ogusa/tests/test_basic.py 41.02% <44.44%> (+1.02%) ⬆️
ogusa/tests/test_execute.py 68.18% <44.44%> (-16.44%) ⬇️
ogusa/tests/test_startyears.py 50.00% <44.44%> (-7.15%) ⬇️
ogusa/tests/test_txfunc.py 65.07% <44.44%> (-1.59%) ⬇️
ogusa/tests/test_user_inputs.py 60.71% <44.44%> (-7.71%) ⬇️
ogusa/tests/test_run_ogusa.py 32.65% <50.00%> (+2.65%) ⬆️
ogusa/tests/test_TPI.py 59.33% <92.30%> (+2.59%) ⬆️
ogusa/tests/test_SS.py 96.29% <92.85%> (+0.10%) ⬆️
ogusa/tests/test_get_micro_data.py 82.66% <100.00%> (+2.36%) ⬆️
ogusa/get_micro_data.py 98.21% <0.00%> (-1.79%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7a52249...a2e446b. Read the comment docs.

@jdebacker
Copy link
Member Author

Hmm, initial commits passed on Travis CI, but I was getting a local failure due with an error message suggesting too many processes were open.

Now working off some examples from this dask distributed issue to try to open up and tear down a dask client separately for each module.

@jdebacker
Copy link
Member Author

@rickecon Full test suite seems to run through well (and much faster) locally. There are some declines in code coverage for instances of functions that are not run on Travis CI and where additional lines were added to pass the dask client. This is ready for you to review.

@rickecon
Copy link
Member

Really nice @jdebacker .

@rickecon rickecon merged commit 62229d0 into PSLmodels:master Apr 25, 2020
@jdebacker jdebacker deleted the test_dask branch August 15, 2021 17:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants