-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Grouped operations (and more) #46
Conversation
fd9d6eb
to
9716dff
Compare
@@ -6,7 +6,7 @@ | |||
|
|||
|
|||
@lazycompile(njit) | |||
def do_mean(pixels, z_pixels, num_zones, nodata, z_nodata, dtype="float64"): | |||
def do_mean(pixels, z_pixels, num_zones, nodata, z_nodata, out_dtype=np.float32): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why this change is necessary? don't see any matching call-site changes
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I see why. Usually this problem of passing extra args to dask graph is best done with functools.partial(my_func, **fixed_named_args_that_might_alias)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There were a couple of problems with the previous implementation:
dtype
is already a keyword tomap_blocks
, so the naming is ambiguous when passed in through the accessor (which could be done withfunctools.partial
I suppose.- second problem was numba: when passing in a dtype with string notation it would fail compiling. Not sure if that would have been solved by using
functools.partial
, but passing the object works fine
f33362d
to
873a145
Compare
OK this is now ready to be merged. Quick summary: Additions
Improvements
Unrelated changes
|
dc42dd2
to
fc1ec14
Compare
Updated summary: Additions
Improvements
Unrelated changes
|
This PR came from improvement work done for pre-calculating SPIs and such for admin extractions.
Most notably this PR adds:
gammastd_grp
for grouped SPIs, that can be calculated by passing an array with group indices to the.algo.spi
accessormean_grp
grouped means (accessible throughalgo.mean_grp
accessor)rolling
that allows for rolling computations over a defined window. Currently,rolling.sum
is implementedzonal.mean
which prevented the definition of output datatype