Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: obs having QC=6 do not appear in link_obs pictures #558

Open
kdraeder opened this issue Oct 12, 2023 · 0 comments
Open

bug: obs having QC=6 do not appear in link_obs pictures #558

kdraeder opened this issue Oct 12, 2023 · 0 comments
Assignees
Labels
Bug Something isn't working Discussion Requires team discussion Minor Minor functionality, non-critical data; easy fix

Comments

@kdraeder
Copy link
Contributor

Describe the bug

If an ob has an "incoming" QC value that denotes "do not use",
then filter assigns the DART QC = 6 and assigns MISSING_R8 to the data values.
Users might expect that the ob will appear in Matlab diagnostic pictures, with its associated QC,
in the same way that an ob which fails the forward operator appears in the pictures with QC=4 associated with it.
But the QC=6 ob is not displayed.

This happens because obs_seq_to_netcdf uses the MISSING_R8 to assign NetCDF's missing_value
to QC=6 obs in epoch.### (aka obs_seq_final.nc) This is where control switches from QC to data value.
Read_obs_netcdf.m uses the NetCDF missing_values to assign Matlab NaNs,
which are not displayed in the link_obs pictures.

Error Message

None. Users may or may not notice that some obs are not displayed.

Which model(s) are you working with?

Any

Version of DART

v10.8.2, but this behavior extends back to at least 2019.

Have you modified the DART code?

No

Build information

This will happen on any machine.

Discussion

I believe that the general opinion is that users should be able to choose whether QC=6 obs should be displayed.
In real obs assimilations there may be so many that they interfere with seeing the assimilated obs.
In OSSEs (and real obs) it may be valuable to see the them, at least some of the time.
The minimal fix appears to be adding a namelist variable to obs_seq_to_netcdf.nml
to control whether those obs are displayed and making the QC be the controlling variable
instead of the data value. But it would be more convenient for users to put the control
in the Matlab scripts, so that obs_seq_to_netcdf doesn't need to be rerun to change the display mode.

When displaying them, having a data value of MISSING_R8 would be awkward (color scales, etc.),
so we may want to preserve the original value if there is one, or assign a value that can be plotted
in a way that identifies it as QC=6. That might be the right context for using MISSING_R8 in the data.
This might require a separate call to scatter on the same plot with just the QC=6 points.

There are probably other considerations that I'm missing.

@kdraeder kdraeder added Bug Something isn't working Discussion Requires team discussion Minor Minor functionality, non-critical data; easy fix labels Oct 12, 2023
@kdraeder kdraeder self-assigned this Oct 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Discussion Requires team discussion Minor Minor functionality, non-critical data; easy fix
Projects
None yet
Development

No branches or pull requests

1 participant