Write NaN into HDF5 instead of None for metadata
#82
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Updating branch `docker` in this fork to date
Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov>
Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
…sujeong/COMPASS into eap_correction_into_s1_geocode_slc
Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
Co-authored-by: Liang Yu <liangjyu@gmail.com>
seongsujeong
added a commit
to seongsujeong/COMPASS
that referenced
this pull request
Feb 6, 2023
* Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * write NaN instead of None Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com>
seongsujeong
added a commit
that referenced
this pull request
Mar 16, 2023
* Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * Write `NaN` into HDF5 instead of `None` for metadata (#82) * Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * write NaN instead of None Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> * Apply corrections in geocodeSlc module (#78) * Add Boolean flag to acitivate/disactivate LUT corrections * Feed LUT corrections to geocode SLC * Save LUTs in CSLC product when computed, skip otherwise * Remove trailing space * Add cumulative LUT function * Modify CSLC validation script (#77) * Throw error when percentage of pixels is above threshold * Address comments on threshold and relative difference: * Specify threshold in the print message. Suppress numpy warnings * Use masked array; modify failed pixel percentage computation * Use masked array to compute NaNs * Correct division by reference pixels * More verbose way to compute absolute difference * compare raster per pixel use masked arrays assign thresholds to variables report on both real and imag % fails Co-authored-by: Liang Yu <liangjyu@gmail.com> * make common burst filtering optional in stack processor (#60) * make common burst filtering optional in stack processor * also add gix for bad function name * Update dependencies (#88) * Add pandas dependency * add lmxl * Include default values in runconfig written to HDF5 metadata (#87) * add default options to metadata runconfig str * Update src/compass/utils/runconfig.py Co-authored-by: Scott Staniewicz <scott.stanie@gmail.com> * Unit test for IONEX functionalities (#80) * Unit test for IONEX functionalities * Add ionex unit test data * adopt pytest fixtures * Allocate variables in conftest * rewrite Docker command * Test with pytest in circleci docker - moved copy of source after miniconda install * fix tec URL * ionospheric correction - working version * `ionex_file` added into `dynamic_ancillary_file_group` in runconfig * Interface revision for iono / tropo corrections; test code removed * `ionex_path` added * shortcut to ionex path in cfg * reverting the changes in parameters for calling `az_fm_rate_mismatch_mitigation` * ionospheric correction writeout to CSLC * removed unnecessary lines * fixing codacy issue * PEP8 issue * updating runconfig template for CI * revision on runconfig scheme and defaults * PR comments addressed * docstring revised * revision on the description in corrections * addressing comments in the 2nd round review * PEP8 --------- Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> Co-authored-by: Scott Staniewicz <scott.j.staniewicz@jpl.nasa.gov>
seongsujeong
added a commit
to seongsujeong/COMPASS
that referenced
this pull request
Mar 31, 2023
* Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * Write `NaN` into HDF5 instead of `None` for metadata (opera-adt#82) * Mod on Dockerfile for cslc_s1 point release and few improvement * Docker image size optimized * Removing the commands in the header comment * Initial commit after branch-out * minor fix; metadata writeout * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/elevation_antenna_pattern.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Comments on PR addressed; docstring for `apply_eap_correction` fixed * comment clarification * comment clarification * addressing codacy issues * Applying EAP correction (or skip) before `range_split_spectrum` * addressing codacy issue * metadata field name changed * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Update src/compass/s1_geocode_slc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * converting burst ID to `str` * using EPSG from geogrid, not from input DEM * Revert "Merge branch 'eap_correction_into_s1_geocode_slc'" This reverts commit d2d5971, reversing changes made to b5e9341. * adding azimuth FM rate mismatch into lut.py * docstring fixed; variable name changed * docstring for file added; raise error when DEM is not valid * Update src/compass/utils/lut.py Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * updates on local * returning az_lut as `isce3.core.LUT2d()` code based on the suggestion from @LiangJYu * docstring revised * removing redundant renaming of the variables * Dockerfile for beta release * Specfile updated * specifile updated * tag on beta.Dockerfile updated * specifile updated * typo fix * beta.Dockerfile updated * updates on `s1_geocode_slc.py` * but fix * updates on beta.Dockerfile * Dockerfile renamed * untested writing az/rg correction LUT to hdf5 moved EAP to corrections group * restructed HDF5 LUT data not working yet * fix doppler return type clean up debugs * fix paths * fix correction group organization * camel to snake fix correction spacing * restore s1_burst_metadata reorg doppler corrections * PEP8 and syntax fixes * fix correction grouping * specfile updated * Specfile updated * s1-reader version updated * Added `0.2` into `version.Tag` * remove radar grid usage from geo grid orbit item names all snake correction reorg * fix CSLC raster validation * fix typo Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * docstring revised * ref -> reference w/r/t _epoch add spacing datasets add metadata path comparision in validation * fix burst border _ID to _id added more metadata to enable RadarGridProduct reconstruction * Separate bistatic and azimuth FM rate * change function return unpacking * add all parameters needed to reconstruct burst object * wrap value assignment with try+except move helper to helpers up top for visibility * burst DB now static correct h5 write exception type * fix burst.center type * fix writing empty burst polynomials * fix misformed poly1d items * rename geocoding interpolator item * move runconfig * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * fix product name * add az fm rate mismatch into `h5_helpers.py` * reverting s1_geocode_slc.py * reverting `lut.py` * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * Update src/compass/utils/h5_helpers.py Co-authored-by: Seongsu Jeong <sjeong.kr@gmail.com> * spelling error * fix descriptions * Version number changed * add `validate_cslc.py` to `setup.cfg` * Flexibility on Docker's entrypoint to run the validation script * PEP8 * add entrypoint to `validate_cslc.py` * fix on `setup.cfg` * Update src/compass/utils/validate_cslc.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * Change version for s1-reader * Removing default value for `dem_path` * Update src/compass/utils/lut.py Co-authored-by: Liang Yu <liangjyu@gmail.com> * release date updated for `0.1.3` * Change base Docker image into Oracle Linux * typo fix * write NaN instead of None Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> * Apply corrections in geocodeSlc module (opera-adt#78) * Add Boolean flag to acitivate/disactivate LUT corrections * Feed LUT corrections to geocode SLC * Save LUTs in CSLC product when computed, skip otherwise * Remove trailing space * Add cumulative LUT function * Modify CSLC validation script (opera-adt#77) * Throw error when percentage of pixels is above threshold * Address comments on threshold and relative difference: * Specify threshold in the print message. Suppress numpy warnings * Use masked array; modify failed pixel percentage computation * Use masked array to compute NaNs * Correct division by reference pixels * More verbose way to compute absolute difference * compare raster per pixel use masked arrays assign thresholds to variables report on both real and imag % fails Co-authored-by: Liang Yu <liangjyu@gmail.com> * make common burst filtering optional in stack processor (opera-adt#60) * make common burst filtering optional in stack processor * also add gix for bad function name * Update dependencies (opera-adt#88) * Add pandas dependency * add lmxl * Include default values in runconfig written to HDF5 metadata (opera-adt#87) * add default options to metadata runconfig str * Update src/compass/utils/runconfig.py Co-authored-by: Scott Staniewicz <scott.stanie@gmail.com> * Unit test for IONEX functionalities (opera-adt#80) * Unit test for IONEX functionalities * Add ionex unit test data * adopt pytest fixtures * Allocate variables in conftest * rewrite Docker command * Test with pytest in circleci docker - moved copy of source after miniconda install * fix tec URL * ionospheric correction - working version * `ionex_file` added into `dynamic_ancillary_file_group` in runconfig * Interface revision for iono / tropo corrections; test code removed * `ionex_path` added * shortcut to ionex path in cfg * reverting the changes in parameters for calling `az_fm_rate_mismatch_mitigation` * ionospheric correction writeout to CSLC * removed unnecessary lines * fixing codacy issue * PEP8 issue * updating runconfig template for CI * revision on runconfig scheme and defaults * PR comments addressed * docstring revised * revision on the description in corrections * addressing comments in the 2nd round review * PEP8 --------- Co-authored-by: Seongsu Jeong <seongsu.jeong@jpl.nasa.gov> Co-authored-by: vbrancat <virginia.brancato@jpl.nasa.gov> Co-authored-by: Liang Yu <liangjyu@gmail.com> Co-authored-by: Heresh Fattahi <hersh.fattahi@gmail.com> Co-authored-by: Zhang Yunjun <yunjunzgeo@gmail.com> Co-authored-by: Scott Staniewicz <scott.j.staniewicz@jpl.nasa.gov>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I have came across the case that writing
Noneinto HDF5's dataset causes an error.This can be a problem when processing old S1 data because they do not have some values in noise annotation (e.g. azimuth_first_azimuth_line).
BurstNoiseclass fills those missing values asNone, andcorrections_to_h5group()inh5_helpers.pywould attempt to writeNoneinto the output HDF5 file.This fix will write
np.naninstead ofNoneto avoid the error. Also the descriptions for the corresponding metadata are modified accordingly.