Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add xarray/dask bilinear resampling #519

Merged
merged 7 commits into from
Dec 11, 2018

Conversation

pnuu
Copy link
Member

@pnuu pnuu commented Nov 26, 2018

This PR adds bilinear interpolation by interfacing the corresponding functionality from pyresample. Note also pytroll/pyresample#148 which is actually the bottleneck killing the performance.

@pnuu pnuu added enhancement code enhancements, features, improvements component:resampling PCW Pytroll Contributors' Week work in progress labels Nov 26, 2018
@pnuu pnuu self-assigned this Nov 26, 2018
Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The source lon/lat data masking shouldn't be needed anymore (see specific comment). Also could you add the bilinear method to the docstring at the top of the module (there is a CSV table).

satpy/resample.py Outdated Show resolved Hide resolved
@coveralls
Copy link

coveralls commented Nov 26, 2018

Coverage Status

Coverage increased (+0.2%) to 74.837% when pulling 26ae344 on pnuu:feature-xarray-dask-bilinear into 6e5ec68 on pytroll:master.

@codecov
Copy link

codecov bot commented Nov 26, 2018

Codecov Report

Merging #519 into master will increase coverage by 0.21%.
The diff coverage is 97.75%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #519      +/-   ##
==========================================
+ Coverage   74.62%   74.83%   +0.21%     
==========================================
  Files         136      136              
  Lines       18342    18408      +66     
==========================================
+ Hits        13687    13776      +89     
+ Misses       4655     4632      -23
Impacted Files Coverage Δ
satpy/tests/test_resample.py 98.89% <100%> (+0.27%) ⬆️
satpy/resample.py 88.79% <94.11%> (+7.19%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6e5ec68...26ae344. Read the comment docs.

satpy/tests/test_resample.py Show resolved Hide resolved
satpy/tests/test_resample.py Outdated Show resolved Hide resolved
@mraspaud mraspaud added this to the v0.11 milestone Nov 28, 2018
Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for updating the documentation. My only other request, maybe on a longer term timeline, would be to subclass from the kdtree resampler and try to use its caching functionality if possible.

@pnuu
Copy link
Member Author

pnuu commented Nov 28, 2018

Ok, I had a quick look and sub-classing shouldn't be very complicated. I'm also thinking the cache handling could be in the BaseResampler class and inherited from there. But as was pointed out in Slack, not all resamplers can do caching, so would need to be careful that this doesn't cause problems. Maybe another issue/PR for the sub-classing?

@djhoese
Copy link
Member

djhoese commented Nov 28, 2018

If you can move the caching to the base resampler but also provide kwargs somewhere (resamplers only accept two positional arguments in __init__ right now) for turning certain caching on/off that would be good.

Should cache_dir be an init argument? Or a precompute argument only?

I think disk-caching should be off by default, memory caching on by default.

@pnuu
Copy link
Member Author

pnuu commented Nov 28, 2018

I agree that disk-caching should be off by default, like it is now. And memory-caching can be on by default, but I'd like a way to disable it: it will be necessary for large source/target areas and different resolutions and/or satellites.
For the rest, let's have another issue/PR for them, "refactor resamplers" or something.

@mraspaud mraspaud merged commit 305d9f6 into pytroll:master Dec 11, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:resampling enhancement code enhancements, features, improvements PCW Pytroll Contributors' Week
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make bilinear interpolation dask/xarray friendly
5 participants