-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ionosphere correction #102
Ionosphere correction #102
Conversation
Updating branch `docker` in this fork to date
Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov>
Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
…sujeong/COMPASS into eap_correction_into_s1_geocode_slc
Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you @seongsujeong. I did not have time to test the PR, unfortunately. However, I provided some comments. Do you mind sharing the TEC file for Rosamond. This might make testing much faster. Thank you :)
@@ -17,6 +17,8 @@ runconfig: | |||
dynamic_ancillary_file_group: | |||
# Digital elevation model | |||
dem_file: | |||
# IONEX file for ionosphere correction | |||
ionex_file: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ionex is a file format, let's rename this to something more descriptive e.g. tec_file
. Also let's add documentation of what this file is e.g. GNSS total electron content file in Ionex format
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what was done in nisar
src/compass/utils/lut.py
Outdated
descr = ['geometrical doppler', 'bistatic delay', 'azimuth FM rate mismatch', | ||
'slant range Solid Earth tides'] | ||
'slant range Solid Earth tides', 'ionospheric delay'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's specify that the ionosphere is along the LOS.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good idea. For consistency, what do you think about adding azimuth
or slant range
for the first three corrections? i.e. geometrical doppler
, bistatic delay
, and azimuth FM rate mismatch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, it is a good idea.
src/compass/utils/lut.py
Outdated
@@ -362,3 +367,55 @@ def resample_set(geo_tide, pts_src, pts_dest): | |||
bounds_error=False, fill_value=0) | |||
rdr_tide = rgi_func(pts_dest) | |||
return rdr_tide | |||
|
|||
|
|||
def ionosphere_delay(sensing_time, wavelength, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The input of the function should be generic. I assume here you need a date to be provided in a certain format not necessarily the burst sensing start.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the suggestion. I agree the placement of this function, and generic input for the time component of this function. For that I've moved the function into iono.py
, and renamed sensing_time
into utc_time
For the data type of utc_time
I think I prefer to keep datetime.datetime
. What do you think?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I like having the data type of utc_time
to be datatime.datetime
(let's clarify it in a comment or in the docstring), what do you think @LiangJYu ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
datetime.datetime
sounds good to me.
src/compass/utils/lut.py
Outdated
|
||
Returns | ||
------- | ||
slant_range_delay: np.ndarray |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's rename this variable to reflect that this is a LOS ionosphere delay
src/compass/utils/lut.py
Outdated
Ionospheric delay in slant range | ||
''' | ||
|
||
if not ionex_path: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is ok to have this check inside the function but we need to discuss what is the default behavior. If a user wants to generate a CSLC-S1 does he/she need to always provide a TEC file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point. In the use case above, I think the user still cat get the CSLC-S1 output without providing TEC file by setting correction_luts.enabled
in runconfig to False
. In that case lut.cumulative_correction_luts()
does not get triggered.
In that case, however, all other LUT-based corrections will be turned off (e.g. bistatic delay, azimuth FM rate mismatch, SET, tropo, etc.) I think we might need to discuss whether someone might want to turn on / off each individual components of the LUT correction.
I decided to put the IONEX file check here because reaching to this point means that the workflow is about to apply ionosphere correction, and TEC file is mandatory in this case.
One thing I am not certain is, whether we would raise an error to stop the processing, or just return zero-filled array and warn the users. My preference is to raise the flag.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's resume the discussion on the default behavior in our next meeting. In the meantime, we do not want the user to get until here if the TEC file has not been provided. I recommend adding a check in geo_runconfig.py
similarly to what has been done here https://github.com/vbrancat/COMPASS/blob/016eb4d762b440ba0609e9a8f5fd495a14e8f55a/src/compass/utils/geo_runconfig.py#L73 so that we stop the SAS immediately if the TEC file is not provided
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With respect to the SAS default behavior, just a heads up, I remember we decided to support an interface where the provided path to the TEC file might be None
. If I remember correctly, we want still apply the other corrections but only apply the ionosphere correction if the path to the TEC file is provided in the runconfig
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@vbrancat Thanks for the suggestion and information. I've added the check in geo_runconfig.py
for TEC file. Also, in iono.ionosphere_delay()
, I've modified it to return a zero-filled array when tec_file
is None
to keep the workflow going without ionosphere correction.
if not tec_file:
print('"tec_file" was not provided. '
'Ionosphere correction will not be applied.')
return np.zeros(lon_arr.shape)
src/compass/utils/lut.py
Outdated
''' | ||
|
||
if not ionex_path: | ||
raise RuntimeError('LUT correction was enabled, ' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Irrespective of the setting for the corrections, if ionex_path
is not provided, we cannot perform the correction. The error message should reflect that and be independent of the correction settings. Also, let's check directly that the path exists
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that the error message should be independent to the correction setting. Please take a look how the revised message looks like.
For file existence check, I think we might better to keep the current pattern because os.path.exists(None)
will raise an error like below. Also this early-return pattern will make it easier to read.
os.path.exists(None)
Traceback (most recent call last):
File "", line 1, in
File "/Users/jeong/miniconda3/lib/python3.9/genericpath.py", line 19, in exists
os.stat(path)
TypeError: stat: path should be string, bytes, os.PathLike or integer, not NoneType
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed as mentioned in the comment above.
src/compass/utils/lut.py
Outdated
ionex_val = ionex_val.reshape(lon_arr.shape) | ||
|
||
freq_sensor = isce3.core.speed_of_light / wavelength | ||
electron_per_sqm = ionex_val * 1e16 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add a comment why we need to do this conversion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comments for K
and electron_per_sqm
was copied from nisar. Also the name of the constant was capitalized to show that this is constant.
src/compass/utils/h5_helpers.py
Outdated
@@ -608,6 +608,9 @@ def corrections_to_h5group(parent_group, burst, cfg, rg_lut, az_lut, | |||
Meta('los_solid_earth_tides', ds.GetRasterBand(4).ReadAsArray(), | |||
f'Solid Earth tides (range) {desc}', | |||
{'units': 'meters'}), | |||
Meta('ionospheric_delay', ds.GetRasterBand(5).ReadAsArray(), | |||
f'Ionospheric delay {desc}', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's add the direction of the ionosphere delay
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not much to add beyond what @vbrancat already commented on.
@@ -17,6 +17,8 @@ runconfig: | |||
dynamic_ancillary_file_group: | |||
# Digital elevation model | |||
dem_file: | |||
# IONEX file for ionosphere correction | |||
ionex_file: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what was done in nisar
src/compass/utils/lut.py
Outdated
@@ -362,3 +367,55 @@ def resample_set(geo_tide, pts_src, pts_dest): | |||
bounds_error=False, fill_value=0) | |||
rdr_tide = rgi_func(pts_dest) | |||
return rdr_tide | |||
|
|||
|
|||
def ionosphere_delay(sensing_time, wavelength, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not immediately relevant to this PR: @seongsujeong @vbrancat what do you all think consolidating ionosphere functionality (and maybe other atmospheric corrections) within ISCE3? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some more small comments
src/compass/utils/lut.py
Outdated
@@ -27,6 +27,8 @@ def cumulative_correction_luts(burst, dem_path, | |||
Sentinel-1 A/B burst SLC object | |||
dem_path: str | |||
Path to the DEM file | |||
tec_path: str | |||
Path to the TEC file for ionosphere correction |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's specify the file format for the expected TEC file
src/compass/utils/lut.py
Outdated
@@ -44,16 +46,17 @@ def cumulative_correction_luts(burst, dem_path, | |||
and slant range | |||
''' | |||
# Get individual LUTs | |||
geometrical_steer_doppler, bistatic_delay, az_fm_mismatch, [tide_rg, _]= \ | |||
geometrical_steer_doppler, bistatic_delay, az_fm_mismatch, [tide_rg, _], ionosphere = \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let's identify the variable as los_ionosphere
src/compass/utils/lut.py
Outdated
LUT spacing along x/slant range in meters | ||
ystep: int | ||
az_ystep: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does not reflect the name of the function variable
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
az_ystep: float | |
az_step: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for catching this. That was a typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM nothing to add beyond what @vbrancat commented on.
src/compass/utils/lut.py
Outdated
LUT spacing along x/slant range in meters | ||
ystep: int | ||
az_ystep: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
az_ystep: float | |
az_step: float |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks @seongsujeong
* Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * Write `NaN` into HDF5 instead of `None` for metadata (opera-adt#82) * Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * write NaN instead of None Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> * Apply corrections in geocodeSlc module (opera-adt#78) * Add Boolean flag to acitivate/disactivate LUT corrections * Feed LUT corrections to geocode SLC * Save LUTs in CSLC product when computed, skip otherwise * Remove trailing space * Add cumulative LUT function * Modify CSLC validation script (opera-adt#77) * Throw error when percentage of pixels is above threshold * Address comments on threshold and relative difference: * Specify threshold in the print message. Suppress numpy warnings * Use masked array; modify failed pixel percentage computation * Use masked array to compute NaNs * Correct division by reference pixels * More verbose way to compute absolute difference * compare raster per pixel use masked arrays assign thresholds to variables report on both real and imag % fails Co-authored-by: Liang Yu <liangjyu@gmail.com> * make common burst filtering optional in stack processor (opera-adt#60) * make common burst filtering optional in stack processor * also add gix for bad function name * Update dependencies (opera-adt#88) * Add pandas dependency * add lmxl * Include default values in runconfig written to HDF5 metadata (opera-adt#87) * add default options to metadata runconfig str * Update src/compass/utils/runconfig.py Co-authored-by: Scott Staniewicz <scott.stanie@gmail.com> * Unit test for IONEX functionalities (opera-adt#80) * Unit test for IONEX functionalities * Add ionex unit test data * adopt pytest fixtures * Allocate variables in conftest * rewrite Docker command * Test with pytest in circleci docker - moved copy of source after miniconda install * fix tec URL * ionospheric correction - working version * `ionex_file` added into `dynamic_ancillary_file_group` in runconfig * Interface revision for iono / tropo corrections; test code removed * `ionex_path` added * shortcut to ionex path in cfg * reverting the changes in parameters for calling `az_fm_rate_mismatch_mitigation` * ionospheric correction writeout to CSLC * removed unnecessary lines * fixing codacy issue * PEP8 issue * updating runconfig template for CI * revision on runconfig scheme and defaults * PR comments addressed * docstring revised * revision on the description in corrections * addressing comments in the 2nd round review * PEP8 --------- Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> Co-authored-by: Scott Staniewicz <scott.j.staniewicz@jpl.nasa.gov>
This PR adds ionosphere correction feature to COMPASS. Currently I label this PR
work in progress
because there are some points that needs to be coordinated with the pending PRs in COMPASS.Below are few notes:
correction_luts
is enabled but IONEX file was not provided, or does not exist.rdr2geo
when calculating azimuth FM rate mismatch.4.1. Eventually solid earth tide computation (in the PR Solid Earth Tides correction #91 ) will run
rdr2geo
. This run will generatelatitude
,longitude
,height
,incidence angle
, andheading
to be used for other corrections, which are: