Evaluate GMAO conventional obs YAMLs

[ADCollard]

Evaluate the GMAO conventional data YAMLs in the context of the GDASApp.

[ADCollard]

GMAO have shared one of their YAMLs for aircraft. This does not include the bias correction, so we can work on adding that.

[ADCollard]

Looking at the aircraft YAML from GMAO.

There are some additional filters required that are not available in UFO develop.

ObsErrorFactorDuplicateCheck
ObsErrorFactorPressureCheck

There is also a weird error

Test "ufo/ObsFilters/testFilters" failed with unhandled eckit::Exception: Bad Conversion: 
Cannot convert (1.4,1.4,1.4,1.4,1.4,1.4) (List) to double @ 
 (/scratch1/NCEPDEV/global/spack-stack/spack-stack-v1/cache/build_stage/emc.nemspara/spack-stage-eckit-1.20.2-cxvre73ysjwofwoqpbwssnsqeucjlg2h/spack-src/src/eckit/value/Content.cc +75 badConversion)

Without the new filters testing will be limited, but we can look at aircraft bias correction and humidity QC (which is not included by GMAO).

[ADCollard]

Finally got the system working. Air temperature shows significant differences as would be expected from not having bias correction in place:

Final ob errors are fairly close but differ too:

[ADCollard]

Humidity QC is not included in the GMAO YAMLs.
H(x) differences are at the 1.e-8 level:

Something is wrong with the obs error differences:

[ADCollard]

Wind H(x) differences are very small (as expected):


The obs error differences are very large, however. We certainly need to dig.

[ADCollard]

Re-starting this work with updated GMAO YAML files

Test YAMLs are in https://github.com/NOAA-EMC/GDASApp/tree/feature/gmao_eval/parm/atm/obs/testing/swell

[ADCollard]

Differences for sondes

• HofX is very close. ~1.e-9 for spec humidity and 1.e-5K for temperature.
• Some weird plots for errors. Need to dig further.

[PraveenKumar-NOAA]

@ADCollard thanks for the plots. After comment out a few filter lines in sondes yaml, I was able to run the test and my results are very similar to your results except one for the stationPressure obs error plot:

[ADCollard]

@PraveenKumar-NOAA I am glad you are looking at it. Take a look at my YAML here. It runs without needing to delete any filters.

[PraveenKumar-NOAA]

@ADCollard thanks. I will look into this.

[PraveenKumar-NOAA]

@ADCollard I noticed an additional line 'pressure: MetaData/pressure' at few places in your yaml, it worked now without any comment and I produced exactly your results.

[PraveenKumar-NOAA]

@ADCollard after making the following minor changes in your yaml:

1. uncomment line 34
2. uncomment line 35 and replace surface_geopotential_height by surface_geometric_height
3. line 135: replace surface_altitude by surface_geometric_height

I got the following results for the stationPressure:

If we use the GSI da_psfc_scheme instead of UKMO, the results are very similar:

[ADCollard]

Splitting this YAML into one for sondes and one for aircraft.