Write CSLC raster and correction stats to HDF5

[LiangJYu]

This PR:

• writes stats (currently only found in JSON) to HDF5
• adds iono and SET correction stat computations

[LiangJYu]

Red curly brackets in pic below show where stats are in HDF5

[seongsujeong]

LGTM. Maybe @vbrancat can give more comments about the structure.
I've left a small question regarding the correction data to include.

[vbrancat]

It seems we are still generating the JSON file for the stats. Would it be possible to remove it?

[LiangJYu]

It seems we are still generating the JSON file for the stats. Would it be possible to remove it?

Do you prefer:

1. remove JSON generation by default but keep ability to generate JSON
2. remove ability to generate JSON

[vbrancat]

Let's keep the ability of generating the JSON file but let's not produce it

[vbrancat]

also, shall we indicate that stats are actually QA metric by calling them qa_stats?

[LiangJYu]

Let's keep the ability of generating the JSON file but let's not produce it

Currently the HDF5 doesn't include all the QA fields found in the JSON (pixel_classification_percentatges, rfi_info, orbit_info).

What do you think about consolidating all QA items under one quality_assurance group in the HDF5 as detailed below?

science/SENTINEL1/CSLC
├── corrections
├── grids
├── metadata
└── quality_assurance (NEW)
    ├── orbit_info
    │   └── orbit_type
    ├── pixel_classification_percentages
    │   ├── topo
    │   │   ├── percent_layover_pixels
    │   │   ├── percent_shadow_pixels
    │   │   └── percent_combined_pixels
    │   ├── percent_land_pixels
    │   └── percent_valid_pixels
    ├── rfi_info
    │   ├── is_rfi_info_available
    │   ├── rfi_mitigation_performed
    │   ├── rfi_mitigation_domain
    │   └── rfi_burst_report
    └── stats
        ├── corrections
        │   └── <correction_name>
        │       ├── mean
        │       ├── min
        │       ├── max
        │       └── std
        └── grids
            └── <polarization>
                ├── mean
                ├── min
                ├── max
                └── std

[vbrancat]

Yes, I like that. Be aware pixel_classification_percentages that this:

│   ├── topo
  │   │   ├── percent_layover_pixels
  │   │   ├── percent_shadow_pixels
  │   │   └── percent_combined_pixels

should go into the static layers topo product and not in CSLC

[LiangJYu]

Yes, I like that. Be aware pixel_classification_percentages that this:

│   ├── topo
  │   │   ├── percent_layover_pixels
  │   │   ├── percent_shadow_pixels
  │   │   └── percent_combined_pixels

should go into the static layers topo product and not in CSLC

For the sake of consistency, should I add quality_assurance group to the static layer HDF5 with:

1. move topo percentage over from CSLC HDF5
2. add stats for each static layer
3. organization structure below:

science/SENTINEL1/CSLC
science/SENTINEL1/CSLC
├── grids
├── metadata
└── quality_assurance (NEW)
    ├── orbit_info
    │   └── orbit_type
    ├── pixel_classification_percentages
    │   └── topo
    │       ├── percent_layover_pixels
    │       ├── percent_shadow_pixels
    │       └── percent_combined_pixels
    └── stats
        └── static_layers
            └── <layer_name>
                ├── mean
                ├── min
                ├── max
                └── std

[vbrancat]

Absolutely, let's go for it.

[vbrancat]

A couple of comments:

1. let's spell orbit_info with orbit_information and pixel_classification_percentages with pixel_classification
2. For the burst I processed, both the items in pixel_classification are empty. Is this intentional?
3. This is what I see in rfi_info (let's spell rfi_information)

is_rfi_info_available    Dataset {SCALAR}
    Data:
         TRUE
rfi_burst_report         Dataset {SCALAR}
    Data:
         "\000"
rfi_mitigation_domain    Dataset {SCALAR}
    Data:
         TRUE
rfi_mitigation_performed Dataset {SCALAR}
    Data:
         TRUE

Is this intentional? For the RFI domain I was expecting a string.

4. Let's spell stats with statistics and imag with imaginary

[LiangJYu]

2. pixel_classification for both CSLC and static layers are 0 (I assume that's what you mean by empty) because it's a placeholder field with no computation being done to populate.
3. rfi_domain should be a string. Since it's a placeholder, I've switch to an empty string as a placeholder value.

[vbrancat]

Thanks for having addressed my comments.