Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix selection of user defined scanlines #2 #15

Merged
merged 8 commits into from
Mar 19, 2019

Conversation

sfinkens
Copy link
Member

@sfinkens sfinkens commented Nov 12, 2018

Remake of #12

I believe there is a bug in pygac.gac_io.save_gac. User defined scanlines are not selected correctly in the presence of scanlines with invalid lat/lon info:

  1. Choose new temporary start/end lines due to invalid lat/lon info: data = data[temp_start_line:temp_end_line]
  2. Adjust user defined start_line/end_line to new slice
  3. Select user defined scanlines data = data[start_line, end_line]

I believe step 2 is not done correctly. Here is a sketch with the details: pygac_2step_slice.pdf

The consequence for CLARA-A2 is, that we get scanlines beyond the requested range from pygac. As the range was chosen to avoid overlap between consecutive orbits, we re-introduce an overlap of temp_start_line scanlines. I did a quick CLARA-A2 logfile analysis:

Year Platform # of files with temp_start_line > 0 Average temp_start_line Standard Deviation
2011 NK 5520 15 0.16
2011 NL 5493 15 0.12
2011 NN 5509 15 0.44
2011 M2 7507 15 0.18
2011 NP 5505 15 1.89
1984 NC 3 1202 625

In 2011 almost every orbit from every platform is affected, but luckily the average temp_start_line is 15, which is not a serious problem. In 1984, the additional overlap is quite significant, but only very few orbits are affected.
Cloud_cci is not affected at all, because they always process all scanlines.

In the presence of scanlines with invalid lat/lon info, user
defined scanlines are not selected correctly.

- Fix selection of user defined scanlines
- Add tests covering that problem
Set to None if it has been removed due to invalid lat/lon info
Didn't work at the end of a month
Use test timestamps with year and month switch
Since pytroll#11 UTC timestamps are saved to file. They have to be sliced
like the other variables in save_gac().
Set to None if midnight scanline is outside the user defined
scanline range
Use mock to patch methods
@sfinkens sfinkens changed the title Fix scanline selection #2 Fix selection of user defined scanlines #2 Nov 12, 2018
@sfinkens sfinkens mentioned this pull request Feb 25, 2019
@mraspaud mraspaud merged commit 94cfe8e into pytroll:develop Mar 19, 2019
@mraspaud mraspaud added the bug label Jun 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants