Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DNB enhancement compositors #254

Merged
merged 7 commits into from
Apr 16, 2018
Merged

Fix DNB enhancement compositors #254

merged 7 commits into from
Apr 16, 2018

Conversation

djhoese
Copy link
Member

@djhoese djhoese commented Apr 16, 2018

Convert VIIRS DNB compositors to work with xarray and dask. This PR takes the easy road for some of them by running delayed(func) then da.from_delayed. These algorithms/enhancements aren't vectorized so they aren't simple to completely dask-ify. They also involve histogram calculations which make dask-ifying not super useful unless done really really well.

Note the VIIRS true color (or anything using the ratio sharpening from the viirs_sdr reader) fails currently due to a bug in XArray or what I consider a bug (pydata/xarray#2060). My viirs_sdr reader assigns a "coordinates" attribute to the datasets with either [m_longitude, m_latitude] or [i_longitude, i_latitude]. Due to the bug this raises an exception when datasets are concatenated together.

  • Tests added
  • Tests passed
  • Passes git diff origin/develop **/*py | flake8 --diff

@djhoese djhoese added this to the v0.9 milestone Apr 16, 2018
@djhoese djhoese self-assigned this Apr 16, 2018
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) to 61.18% when pulling e62d244 on bugfix-viirs-dnb-dask into 48bbc52 on develop.

1 similar comment
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.06%) to 61.18% when pulling e62d244 on bugfix-viirs-dnb-dask into 48bbc52 on develop.

@coveralls
Copy link

coveralls commented Apr 16, 2018

Coverage Status

Coverage increased (+2.7%) to 63.898% when pulling b29edfa on bugfix-viirs-dnb-dask into 48bbc52 on develop.

(-20037508.34, -10018754.17, 20037508.34, 10018754.17))

comp = ERFDNB('dynamic_dnb', prerequisites=('dnb',),
standard_name='toa_outgoing_radiance_per_'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

E127 continuation line over-indented for visual indent

@codecov-io
Copy link

codecov-io commented Apr 16, 2018

Codecov Report

Merging #254 into develop will increase coverage by 2.65%.
The diff coverage is 88.12%.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #254      +/-   ##
===========================================
+ Coverage    61.24%   63.89%   +2.65%     
===========================================
  Files          111      112       +1     
  Lines        13711    13858     +147     
===========================================
+ Hits          8397     8855     +458     
+ Misses        5314     5003     -311
Impacted Files Coverage Δ
satpy/composites/crefl_utils.py 0% <0%> (ø) ⬆️
satpy/readers/viirs_sdr.py 80.82% <100%> (+0.09%) ⬆️
satpy/tests/compositor_tests/__init__.py 98.48% <100%> (+0.02%) ⬆️
satpy/composites/viirs.py 78.69% <73.75%> (+78.69%) ⬆️
satpy/tests/compositor_tests/test_viirs.py 99.24% <99.24%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 48bbc52...b29edfa. Read the comment docs.

@djhoese djhoese merged commit 1fe7e8c into develop Apr 16, 2018
@djhoese djhoese deleted the bugfix-viirs-dnb-dask branch April 16, 2018 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants