end-to-end test for GNSS RO

[XuanliLi-NOAA]

Test GNSS RO data from bufr ioda-converter to fv3-jedi.

[XuanliLi-NOAA]

The end-to-end test for GNSS RO data can proceed, but we observed an issue. Below is the scatter plot for obs vs. hofx from fv3. It appears that there is an noticeable cutoff for obs > 0.03 rad. Both runs use the same obs file that has been created by GSI observer.

The plot for obs vs. hofx_geoval is shown below.

A check on the difference between the hofx_geoval and hofx_fv3 indicated that this is mainly caused by the missing values of fv3_hofx due to super refraction 2. Here is the plot where we replace the missing hofx_fv3 for SR 2 when obs > 0.3 with hofx_geoval.

To track the cause of the difference in SR2, the diag files were dumped and compared. It seems that the entire virtual temperature field is different. We also see some differences in the vertical layer index. Here are some sample outputs:

The plots of hofx_geoval vs. hofx_fv3 and hofx_geoval vs. difference between hofx_fv3 and hofx_geoval are shown below:

The difference between hofx_geoval and hofx_fv3 ranges from -42% to 20%, which appears to be significant. Could this difference be related to the background fields used for the two runs?

Difference in hofx vs. pressure is shown below:

[XuanliLi-NOAA]

I talked to Hailing, she reported a similar issue (https://github.com/JCSDA-internal/ufo/issues/1374). But the difference that she found was much smaller than ours.

[nicholasesposito]

thanks for putting this together. Is there a way to check the background fields? And do we think this could be a larger UFO issue since it was seen elsewhere?

[XuanliLi-NOAA]

Emily might know better about this. The background that I used for GSI observer to create the obs and geoval files is in this directory: /work2/noaa/da/cmartin/UFO_eval/data/para/output_ufo_eval_aug2021/2021080100/gdas.20210731/18/atmos/.

[emilyhcliu]

@XuanliLi-NOAA Are you saying that the fv3 background is causing the missing values due to the super refraction checks?
If so, then the results are consistent between using Nick's IODA file and the GSI generated Obs file for the HofX_fv3 run. It is the backgound values that trigger the super refraction check and set the HofX as missing in the Hofx fv3 run.

In this case, we need to check the geoval and the fv3 background.
We can write out the geovals from the hofx_fv3 run and then compare them with the geovals from GSI. (Nick already knew how to do this)

For each geoval variables, do the scatter plot (geoval_fv3 and geoval_gsi). Not sure if the data will be too large to plot since these are 3D. The 2D map would be good too.
For each geoval variables (T, q, geopotential height, ...), do the scatter plot by level (2D) between geoval_fv3 and geoval_gsi). We can focus on the levels where the super-refraction occurs.

[XuanliLi-NOAA]

I talked to Hailing; she thinks the difference between the hofx_geoval and hofx_hv3 is too large, and the vertical layer index was computed in ufo. So most likely, this is caused by the background fields. What is the easiest way to output the fv3 background fields like temperature, specific humidity, etc?

[emilyhcliu]

I talked to Hailing; she thinks the difference between the hofx_geoval and hofx_hv3 is too large, and the vertical layer index was computed in ufo. So most likely, this is caused by the background fields. What is the easiest way to output the fv3 background fields like temperature, specific humidity, etc?

Will you be in office tomorrow? If so, please come to my cubical. I will show you how to configure UFO to output geoval from fv3 run. It will be a two-step work. If you are not in tomorrow, I will write you a note about how to write out geoval and send it to you. We can also meet on Wednesday.

[XuanliLi-NOAA]

I'll be in on Wednesday, so please write me a note if that works for you, we can also talk on Wednesday.

[emilyhcliu]

OK, will do.

[XuanliLi-NOAA]

Thank you so much!

[XuanliLi-NOAA]

I used output_diags: true in the yaml file which wrote out some obs diag variables including specific humidity and refractivity. Here are the plot of specific humidity for geoval vs. fv3 and humidity for geoval vs. (geoval-fv3):

It seems that the humidity differences were up to 50%. Differences become larger when the specific humidity becomes small (<1 g/kg).

The plot below shows specific humidity for the hofx missing data points with observation bending angle > 0.03 Rad:

The plots for refractivity difference and virtual temperature difference are shown below:

[XuanliLi-NOAA]

It appears that majority of the hox missing data points occurred over the ocean. The map plot is shown below:

[XuanliLi-NOAA]

The scatter plots for geopotential height and the relative difference are shown below:

It seems that large differences exist when gph < 10 km. For these plots, only Super Refraction 1 was applied.

This is the plot for gph < 10 km.

[XuanliLi-NOAA]

Here are the scatter plots for temperature and the relative differences between geovals and fv3:

This is T difference vs. pressure:

Zoom in

For these plots, only Super Refraction 1 was applied.

[XuanliLi-NOAA]

I'm comparing the geoval file generated by GSI with the one generated from fv3 using " - filter: GOMsaver" in the Yaml file. Here is what I don't understand:
Even the surface altitude could be very different.

I'm using the same obs file for both runs:
diff -qs /work2/noaa/da/xuanli/gps_e2e/ufoeval/2021080100/gps_bend_nbam_sr1/gps_bend_obs_2021080100.nc4 /work2/noaa/da/xuanli/gps_ropp2d/obs/2021080100/gps_bend_obs_2021080100.nc4
Files /work2/noaa/da/xuanli/gps_e2e/ufoeval/2021080100/gps_bend_nbam_sr1/gps_bend_obs_2021080100.nc4 and /work2/noaa/da/xuanli/gps_ropp2d/obs/2021080100/gps_bend_obs_2021080100.nc4 are identical

Here are the surface altitude for data points 45-69:
gev fv3
45 0.0 452.6754
46 0.0 465.80936
47 0.0 483.91232
48 0.0 474.31296
49 0.0 485.89377
50 0.0 513.31824
51 0.0 527.1415
52 0.0 514.2682
53 0.0 486.91464
54 0.0 489.59012
55 0.0 476.0
56 0.0 476.0
57 0.0 562.3009
58 0.0 991.0939
59 0.0 1323.939
60 0.0 1444.4819
61 0.0 1427.8636
62 0.0 1319.2383
63 0.0 1222.1384
64 0.0 1216.601
65 0.0 0.0
66 0.0 23.513435
67 0.0 158.93382
68 0.0 406.11905
69 0.0 697.4912

What could be the reason for such a difference?