Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Daskify Stats #70

Merged
merged 23 commits into from
Jul 8, 2020
Merged

Daskify Stats #70

merged 23 commits into from
Jul 8, 2020

Conversation

bradyrx
Copy link
Owner

@bradyrx bradyrx commented Nov 14, 2019

Description

Cleans up the stats module using simply numpy functions and adds explicit dask support with testing.

Fixes #66, #65, #31, #63

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

To Do List

  • Update docs/API
  • Update CHANGELOG

How Has This Been Tested?

Checklist (while developing)

  • I have added docstrings to all new functions.
  • I have commented my code, particularly in hard-to-understand areas
  • Tests added for pytest, if necessary.
  • I have updated the sphinx documentation, if necessary.

Pre-Merge Checklist (final steps)

  • I have rebased onto master or develop (wherever I am merging) and dealt with any conflicts.
  • I have squashed commits to a reasonable amount, and force-pushed the squashed commits.
  • I have run make html on the documents to make sure example notebooks still compile.

References

Please add any references to manuscripts, textbooks, etc.

@bradyrx
Copy link
Owner Author

bradyrx commented Dec 2, 2019

This example from Ryan could potentially help:
https://nbviewer.jupyter.org/gist/rabernat/30e7b747f0e3583b5b776e4093266114

@aaronspring
Copy link
Collaborator

This looks massive and great. Looking forward to review this once it is ready.

@bradyrx
Copy link
Owner Author

bradyrx commented Dec 13, 2019

I'll tag you to review once it's done! Just doing this in surges when I have energy. But should be really robust and have all the dask support we need.

@bradyrx
Copy link
Owner Author

bradyrx commented Dec 16, 2019

@bradyrx
Copy link
Owner Author

bradyrx commented Jun 29, 2020

from statsmodels acf : ['none', 'raise', 'conservative', 'drop']

esmtools/stats.py Outdated Show resolved Hide resolved
esmtools/stats.py Show resolved Hide resolved
esmtools/stats.py Show resolved Hide resolved
esmtools/tests/conftest.py Show resolved Hide resolved
esmtools/tests/conftest.py Show resolved Hide resolved
esmtools/tests/test_stats_poly_and_corr.py Show resolved Hide resolved
esmtools/timeutils.py Outdated Show resolved Hide resolved
bradyrx added 11 commits July 7, 2020 15:50
add docstrings through utils

add DAYS_PER_YEAR constant
Consolidate regression testing suite.

use parametrize to compress testing even more with various time types
* update xarray decorator to is_xarray, and other small dev fixes

move get_dims to utils

move fixtures to conftest

add testing for linear_slope
only show units warning for slopes

remove has_dims
automatically convert slopes by factor
add timeutils accessor testing

add test for match_nans

add tests for missing data in x axis
@bradyrx bradyrx merged commit 9239fa3 into master Jul 8, 2020
@bradyrx bradyrx deleted the dask_stats branch July 8, 2020 21:23
@bradyrx
Copy link
Owner Author

bradyrx commented Jul 8, 2020

Went ahead and merged @aaronspring, since you reviewed this extensively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add NaN management logic for linear regressions, etc.
2 participants